

NASA 

Reference 

Publication 

1253 


1992 




National Aeronautics and 
Space Administration 

Office of Management 

Scientific and Technical 
Information Program 




Reliability Training 


Edited by 
Vincent R. Lalli 
Lewis Research Center 
Cleveland, Ohio 


Henry A. Malec 
Siemens Stromberg-Carlson 
Albuquerque, New Mexico 


& 





Preface 


What Does Reliability Mean? 

Systems. . . 

The word “reliability” applies to systems made up of people, machines, and written 
information. 

A system is reliable that is, has good reliability— ; f the people who need the system can 
depend on it over a reasonable period of time. People can depend on a system if it reasonably 
satisfies their needs. These statements are purposely somewhat vague because quantifying 
them for any particular situation is a big task in itself. 

People. . . 

Several kinds of people are involved in a system, and they have different views of it. 
Some people rely on the system, others help to keep the system reliable, and still others do 
both. For example, consider an automatic grocery checkout system. The people involved are 

• The owners, who bought the system 

• The store manager, who is responsible for the system’s operation 

• The clerk, who actually operates it 

• The repair person, who keeps it working 

• The customer, who is being waited on 

Machines. . . 

A system can comprise several kinds of machines. A grocery checkout system has 
mechanical parts, electrical parts, and electronic parts. An automobile has chemical parts 
(fuel), liquid parts (hydraulic fluid for brakes), mechanical parts (engine, transmission, 
wheels), electrical parts (wiring, lights), electronic parts (ignition system, radio, engine 
controls), structural parts (body, frame, wheels, seats), miscellaneous parts (windows, 
windshield wiper blades), and many parts that can be classified in several wavs (e a 
the fuel). 1 

Written Information. . . 

Several kinds of written information are important to the way people rely on a system; 
for example, 

• The sales literature that led the owner to buy the system 

• The specifications for the system 

• The detailed manufacturing drawings 

• The software, programs, and procedures 

• operating instructions to the people who actually operate the system 

• The repair instructions to the people who keep the system running and fix its parts when 
it fails 

• The supply instructions so that people know what kind of repair parts should be made 
and stocked 
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• The instructions to the machine, especially computer programs, which are so vital to 
so many machines 

• The inventory control to restock goods 

Reliability. . . 

People rely on systems 

• To do useful or amusing things for them 

• To do no unintentional harm to users, bystanders, property, or the environment 

• To be reasonably economical to own and to fix 

• To be safe to store or dispose of 

• To accomplish their purposes without failure 


What Does Reliability Engineering Mean? 

Reliability engineering means doing special tasks while a system is being planned, 
designed and developed, manufactured, used, and improved. These special tasks are over 
and above the usual engineering and management tasks and are needed to ensure that the 
people involved in these usual tasks pay attention to all important details. These tasks ensure 
that the people who rely on the system will not be let down— not only when it is new, 
but also as the system gets older, worn, and repeatedly fixed. 


Why Do We Need Reliability Engineering? 

We, as users of technology, have always needed reliability engineering, but the separate 
discipline of reliability engineering has developed only since the 1940's. Before the industrial 
revolution most of the reliability detail* were handled by the individual workers for relatively 
simple machines, products, and tools. But shoddy goods were produced— wheels that broke 
too soon, farming implements that were not dependable, wood that rotted before its time. 

Technology is changing rapidly. Systems are now large and complex. Companies that 
produce these systems must likewise be large and complex. In such situations, many 
important details— the kinds that affect reliability— slip by unnoticed in the press of getting 
things done on time and at an affordable cost. The telephone and electric power utilities 
and the military were among the first to see the need for a separate reliability discipline. 
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Chapter 1 

Introduction to Reliability 


This perspective on the past, present, and future of reliability 
was prepared by Mr. Kam L. Wong. It was adapted from a 
keynote speech he gave at the 1982 European Conference on 
Electrotechnics. 

Ever since the need for improved reliability in modern 
systems was recognized, it has been difficult :o establish an 
identity for reliability engineering. Attempts to separate out 
an independent set of tasks for reliability engineering in 
the 1950’s and 1960’s resulted in the development of 
applied statistics for reliability and a large group of tasks for 
management. However, most of these tasks are in truth not 
reliability engineering tasks. Although much of the engineering 
work done in the name of reliability pertains to basic design, 
field failures in a well-designed system come from defects 
(flaws) that remain in the system after delivery and not from 
the basic design. Defect (flaw) control is the key to reliability. 
The traditional reliability tasks for a project are still important 
and should still be performed by reliability engineers. A new 
direction for system reliability engineers should be to act as 
dynamic synthesizing feedbacks— identifying and ranking 
flaws and stresses, determining flaw failure mechanisms, 
and explaining flaw control techniques to those responsible 
for design, manufacturing, and support planning. Reliability 
engineers and basic engineers must work closely together to 
create a synergistic effect for achieving ever higher reliability. 

For the purpose of this chapter reliability engineering is 
defined as a branch of engineering devoted to preserving the 
required performance of a system operating under the 
stipulated conditions for the time period of interest within a 
set of constraints such as cost and weight. This formidable- 
sounding definition means that reliability engineering is a 
branch of engineering for making things work as advertised. 
Such a nebulous definition has made it difficult to establish 
an identity for reliability engineering. 

This chapter identifies traditional reliability disciplines. The 
one reliability discipline excluded from this discussion is 
management. Although management is important, especially 
with the contemporary awareness ot Japanese productivity and 
the continuing quality and reliability of Japanese products, it 
is outside of the scope of this discussion. We concentrate on 
the engineering and technical aspects of reliability, in 1979 
a paper (ref. 1-1) w, 5 published that used the number of 


published papers and the number of their pages as 
measurement indexes to describe the development of reliability 
and maintainability disciplines. This chapter does not use the 
same types of indexes as the 1979 paper. Instead the rough 
magnitudes of published works in reliability areas will be used 
to estimate the relative emphasis. Furthermore, this chapter 
concentrates on reliability engineering and not on the broad 
field of reliability disciplines. When the term “reliability 
engineering” is used, it is understood to relate to systems 
engineering. To forecast for the future, we need to determine 
the what’s and why’s. If we cannot do that, then, at least, we 
need to establish a trend. Therefore, let us begin with what 
has been done in the name of reliability engineering in the last 
40 years. 


Era of Mechanical Designs 

Before World War II, most equipment was mechanical. A 
failure could usually be isolated to a rather simple part. Of 
course, mechanical systems could be complex and contain 
many interacting parts, but the difficulty in assembling such 
products to sell at a reasonable price precluded building 
complex svstems. Therefore, one generally needed to deal only 
with ra*her simple items. 

Safety, which is closely related to reliability, was a critical 
factor in a piece of equipment. The key to reliable products 
then was safety margins in either stress-strength, wear, or 
fatigue conditions. Most of the efforts toward achieving good 
safety margins were simply considered good engineering 
practices. Therefore, calling a task a reliability effort was not 
meaningful. At times redundancy was used to ensure safety 
such as in multiengine aircraft and large structures. In effect, 
reliability in this era was implied in a product and was 
automatically expected by its users. Buyers usually bought only 
from manufacturers that were well known for producing 
reliable products. The only quantitative measure related to 
reliability and considered in equipment procurement and usage 
was the wearout life of the equipment. After something was 
designed and built, the only efforts expended for reliability 
were inspection and testing. Reliability engineering as such 
did not exist. 










Era of Electron Tubes 

The availability of electron tubes opened the way to rapidly 
increasing complexity of equipment, both in functions and 
parts counts. By the end of World War II the state of the an 
was growing much more qjickly in electronics than in 
reliability engineering. The gap between technology and 
reliability in electronic equipment was beginning to be felt by 
the U.S. military. Why should electr nic equipment present 
a greater reliability problem than earlier mechanical systems? 
First, the heart of electronic equipment then was the electron 
tube. An electron tube is a complex device in itself. It is an 
assemblage of many small parts. Material purity— glass, glass 
seal, and cathode— is critical. Thus, an electron tube is not 
highly reliable to begin with. Although it was good enough 
for use in a five-tube radio, the chance of failure went up 
exponentially when complexity increased. Therefore, complex 
digital or analog electronic equipment can have low reliability. 

A complex function could be performed rather easily 
by a piece of electronic equipment constructed by repetiti/c 
standard assembly methods from a large number of mass- 
produced, reasonably priced electron tubes and passive parts. 
Using purely mechanical devices to perform such complex 
functions was not economically feasible. The economics of 
production that enabled economical manufacture of complex 
electronic equipment was also the major driving force for low 
reliability. Assume that a part will sell for a fixed price in the 
marketplace. If a company can spend 10 percent more money 
to gain 15 percent higher production yield, the additional 
spending will give the company more profit. However, if 15 
percent more money will produce a yield gain of only 10 
percent, it may not be profitable to spend the money. Thus, 
in mass production there is a point where the company should 
not put more money into improving production yield. Although 
a quantitative relationship between reliability and yield has not 
been established, low-yield parts probably have low reliability. 
If the number of visible defects (flaws), which cause rejects, 
is high, it is logical that invisible defects will also be high. 
Although these defects are invisible at the time of manufacture, 
they cause failures during equipment usage. 

Total sales was another factor that kept manufacturers from 
improving reliability; if the parts were more reliable, fewer 
replacement parts would be sold. There is no fiscal incentive 
to improve reliability unless the customer complains or a 
competitor’s product demonstrates much higher reliability for 
the same cost. From an economic viewpoint we really should 
not expect the reliability of electronic equipment to improve 
unless a basic improvement in manufacturing process is made 
at no increase in manufacturing cost. Fortunately, this does 
happen, so that reliability generally does improve with r«i^rwi»r 
time. However, the public usually does not wish to pay much 
more for additional reliability. A spare might still be the best 
method for achieving high reliability even in critical operations 
like broadcasting, where using redundant transmitters solves 
the problem. Because of their need to maintain strike capability 


and minimize logistic supplies, the military is most sensitive 
to the reliability problem. 

During the 1940’s the U.S. military promulgated the joint 
Army and Navy (JAN) standards for parts ,.nd established the 
Vacuum Tube Development Committee. By 1946 the airlines 
had set up a study for the development of better electron tubes. 
Later, Aeronautical Radio, Inc., and Cornell University did 
extensive analyses on defective electron tubes. About 1950, 

Vitro Laboratories and Bell Telephone Laboratories also 
pursued studies on failed parts, and the U.S. Department of 
Defense established an ad hoc committee on reliability that 
became the Advisory Group on the Reliability of Electronic 
Equipment (AGREE) in 1952. This group published its 
monumental report in 1957. In the meantime efforts directed 
toward reliability mushroomed. A few of the many noteworthy 
activities and publications during this explosive developmental 
period are listed in table 1-1 . Each entry has some significance 
in the development of reliability engineering. 

Not reflected in table 1-1 are military specifications, 
standards, and handbooks. Military specifications, standards, 
and handbooks were generated in the United States during the 
1950 s, primarily to improve the understanding of reliability. 

Much of the work that resulted in the publications shown in «. 
table 1-1 was i ided by the U.S. Government. The military 
and the Government’s pushing gave birth to reliability 
engineering. Their specifications required that various tasks , 
be done (see fig. 1-1) by an independent system engineering 
group. Whether the product had been designed in a reliable 
manner was the important question. The greater emphasis at 
that time was on the need to make products more reliable by, j 
for example, reliability prediction. Reliability can be predicted f 
by counting parts or by analyzing part stress. Most proposed 
predictions are parts count predictions to provide a model for 
tradeoff studies. 

Various reliability efforts have been grouped into a number 
of categories; manufacturing control, design control, reliability 
methods, failure cause detection, finished item reliability 
control, and flow control. Figure 1-1 depicts how these cate- 
gories have been emphasized through the years. Admittedly, 
the construction of figure I - 1 is rather subjective; its purpose 
is to establish trends, not to classify efforts precisely. Note which 
specific quality and reliability effort emphasis is changing. 

Bear in mind that the amount of effort expended may not be 
proportional to the emphasis, although quite often it is the case. 

For example, wear life is always important. The decrease in 
the design control emphasis does not mean that wear life is 
unimportant. It only reflects that the importance of wear life 
has been well established and that wear life has become a 
standard design control task as part of the design process. ! 


The invention of the transistor in 1948 opened up a new j 
frontier for electronics. The simplicity of semiconducting j 


Era of Semiconductors 
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TABLE l-l. -RECOGNIZED ACTIVITIES AND PUBLICATIONS DURING 
DEVELOPMENTAL PERIOD OF RELIABILITY ENGINEERING 


Date 

Event 

Date 

Event 

July 1949 

Formation of the Professional Group on Quality 
Control. 

May 1955 

Publication of “Sequential Life Tests in the 
Exponential Case.” by B. Epstein and M. Sobel 
in Annals of Mathematical Statistics, vol. 26. 
pp. 82-93 

September 1951 

Publication of “A Statistical Distribution Function 
of Wide Applicability” by W. Weibull in Journal 
of Applied Mechanics, vol. 18, no. 3, pp. 293-297. 

July 1955 

Formation of the Reliability and Quality Control 
Group. 

July 1952 

Publication of “An Analysis of Some Failure 
Data” by D.J. Davis in Journal of American 
Statistical Association, vol. 47, no. 258, pp. 113-150. 

August 1955 

Publication of “Systems Approach to Electronic 
Reliability” by W.F. Lucbbert of U.S. Signal Corps. 

August 1952 

Establishment of the Advisory Group on Reliability 
of Electronic Equipment (AGREE) by the U.S. 
Department of Defense. 

Publication of “A Survey of Current Status of the 
Electronic Reliability Problem,” Rand Research 
Memorandum 1131, by R.R. Carhart. 

September 1955 

Publication of “Handbook of Preferred Circuits, 
Navy Aeronautical Electronic Equipment,” by 
National Bureau of Standards for U.S Navy, 
Naval Weapons Department, in Inst. Radio Eng. 
Proc., vol. 44, pp. 523 -528. 

October 1955 

Publication of Vitro Laboratories Report No 80, 
“Techniques for Reliability Measurements and 
Prediction, Based on Field Failure Data.” 

May 1953 

Publication of “Rudiments of Good Circuit 
Design,” by N.H. Taylor. 

September 1953 

Publication of ’’Life Testing” by B. Epstein and 
M Sobel in Journal of American Statistical 
Association, vol. 48, no. 263, pp. 486-502. 

1956 

Publication of “Reliability Factors lor Ground 
Electronic Equipment,” edited by K. Hcnncy, 
McGraw-Hill, New York. 

— — 

1954 

Publication of monographs on “Electron Tube Life 
and Reliability” by M.A. Acheson. 

November 1956 

Publication of TRII00, “Reliability Stress Analysis 
for Electronic Equipment,” by J.A. Connor ct al. 
of RCA in Trans. Reliability Quality Control, 
vol. PGRQC-9, 

March 1954 

Publication of “NEL Reliability Design Handbook” 
by U.S. Navy Electronics Laboratory. 

June 1957 

Publication of AGREE report “Reliability of Military 
Electronic Equipment” by the Advisory Group on 
Reliability of Electronic Equipment. 

September 1954 

Publication of “Truncated Life Tests in the 
Exponential Case,” by B. Epstein in Annals of 
Mathematical Statistics, vol. 23, p. 639. 

October 1958 

Publication of Technical Report No. 3, U.S. Navy. 
“Statistical Techniques in Life Testing.” by 
B. Epstein. 

November 1954 

First national symposium on quality control and 
reliability in electronics in United States. 

March 1955 

Publication of “RCA Reliability Program and Long 
Range Objective” by C M. Ryerson. 

September 1978 

1 

i 

Formation of the IEEE Reliability Society. 

May 1955 

Publication of “Electronics Reliability: Definition of 
Terms of Interest in Study of Reliability” by G.R. 
Herd ct ai. of Aeronautical Radio, Inc*., in Tram . 
Reliability Quality Control, vol. PGRQC-5. 


devices held promise for much higher reliability. Indeed, 
semiconducting devices ultimately improved equipment reli- 
ability by one to two orders of magnitude over the electron tube 
equivalents. By the mid- 1950's transistors became available 
in sufficient product-on quantities for use in electronic equip- 


ment. In the early I960’s integrated circuits (IC's) were 
invented and now dominate the electronic parts industry. 
During the 1960's reliability methods gained momentum. 
Design review then became a predominant element of 
reliability methods. The total reliability effort has been 
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Figure I- 1 —Distribution of reliability emphasis with respect to calendar year. 


increasing through the years, as shown in figure 1-2. Again, 
classifying tasks to be called reliability engineering is an 
inexact science. Do not attempt to read more than a trend 
indicator in figure 1-2. The launching of Sputnik in 1957 gave 
the world space program a tremendous push. The failure of 
Vangua-d TV3 in the same year and many more U.S. satellite 
failures »n 1958 forced the United States into high gear to strive 
for better reliability. Redundancy then became a life-saving 
tool. Without the application of redundancy in their design, 
many satellites, spacecraft, and to a certain extent boosters 
would have failed. The emphasis placed by the U.S. 
Government on reliability in the 1950’s and early 1960's 
greatly improved equipment reliability. 

While the equipment designers and manufacturers were 
making equipment more reliable, so were the parts suppliers. 
The improvements came from better material purity, better 
process controls, better designs, and new technology. One 
interesting phenomenon developed in semiconducting device 
technology when more complex devices were produced. 
People began to notice that the reliability of semiconducting 
devices was not inversely proportional to the complexity of 
the device, as intuition might have led them to believe. For 
example, a 100-transistor IC is more reliable than a circuit 
constructed from 100 individual transistors. Attempts were 
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Figure 1-2.— Relative buildup of reliability effort in United States. (Related 
efforts such as environmental testing, structural and thermal analysis, parts 
and materials engineering, and standard quality controls are not grouped 
under reliability effort.) 


made to relate reductions in die bonds, wire bonds, seal length 
on the packages, er^. to reliability improvements. But the 
improvements were much greater thon could be accounted for. 
Although not quantitatively proven, the mass production yield 
theory mentioned earlier can be used to explain this phenom- 
enon. In effect, for simple devices the production yield has 







reached a point where additional cost to improve yield would 
not bring sufficient income to increase profit. For complex 
devices tighter process control (tighter in-process inspection) 
pays off in profit because of the much higher rate of yield 
improvement. As was indicated, reliability is positively related 
to production yield. Therefore, when the yield of more 
complex devices is raised to approach that of simpler devices, 
their reliability also approaches that of simpler devices. As 
complex devices become more reliable, more of them will be 
used in equipment, causing the reliability of the equipment 
:o need improvement. 

Many books on reliability statistics were written during this 
growth era of the I950*s and 1960's. Most of them were 
mathematical. In effect, they were books on how to apply 
statistical and probability theories to reliability work. In 
particular, small-sample statistics was the main field of 
application. Most of these books were written for applied 
mathematicians and not engineers. Physics uses much math, 
but applied mathematics used for physics is not physics. The 
same reasoning applies for reliability engineering. Throughout 
the 1960's most of the efforts in developing reliability 
engineering followed the classical method of trying to separate 
out an independent set of disciplines for reliability. In a math- 
ematical analogy, people tried to break down the engineering 
function into orthogonal functions so that each orthogonal term 
could be dealt with individually in the hope of successfully 
recombining all the terms later. Through the years the 
reliability engineer provided a check on the design and process 
control engineers to improve the product's reliability. Like 
an electrical or mechanical engineer, the reliability engineer 
should perform an independent systems engineering function. 


During the heyday of reliability activity a small group of 
engineers recognized that really improving reliability meant 
eliminating the source of failure. This led to the calling of 
the first physics of failure symposium in 1962 (ref. 1-3). 
Since then, much work has been done to investigate failure 
mechanisms. Papers have been presented every year in follow- 
on symposia on the subject. Also, parts screening was be- 
coming a must. The issuance of MIL-STD-883 (ref. 1-4) in 
May 1968 set the tone for microcircuit screening to the present. 
Some real reliability engineering was being done. 

Before proceeding further, consider “The Tale of Two 
Failures." A semiconductor diode developed a short. Analysis 
shjwed that a surge voltage was occurring occasionally that 
exceeded the breakdown voltage of the diode and was burning 
it up. It w as a problem of stress exceeding strength. Let us 
call this a type I failure. A transistor suddenly stopped 
functioning. Analysis showed that aluminum metalization 
opened at an oxide step on the chip. The opening was 
accelerated by the neckdown of the metalization at the step. 
This failure was caused by a manufacturing flaw. In the 
classical terminology this is a random failure. Let us call this 
a type II failure. These two failure types are shown in 
figure I -3. Until now, most of the design control efforts shown 
in figure 1 - 1 have been aimed at the type I failure (i.e., stress 
exceeding strength). Such design controls are important. For 
example, much equipment still has inadequate design, such 
as undercooling leading to overheating, even though cooling 
methods are well known. Designers need only to design 
according to standard methods to provide adequate designs. 
However, most equipment failures in the field bear no relation 
to the results of reasonable stress analyses during design. These 


Period of Awakening 

By the 1970's the implementation of reliability programs 
had become routine in developing equipment for the U.S. 
Government and the military. Basically, the reliability 
programs ensured that certain good engineering practices were 
carried out and provided a reliable product to the customer. 
However, equipment still continued to fail, although at a lower 
rate. Design reviews were helpful, but more was needed. With 
the tight funding situation, the benefits derivable from various 
reliability program elements were questioned, This encouraged 
tailoring the reliability program to the need; that is, doing only 
what gives a high return and not everything in the specification, 
A way to alleviate the customer's repair cost problem was to 
let the manufacturer share the burden. This led to the push 
for reliability improvement warranties (RIW’s). There were 
many ways of implementing RIW's. Some were simply 
warranties such as those on car batteries and household 
appliances (ref. 1-2). These changes were mainly changes iri 
management emphasis; there was really no engineering 
involved, 
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(a) Type I failures (a design margin problem on stress/st ccngth. fatigue, 
and wear), 

<b) Type II failures (a flaw problem). 

Figure 1-3,— Two types of failure. 
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failures are type II (i.e., those caused by built-in flaws). It 
has become evident that flaws are what must be dealt with. 

Flaws have long been recognized as the cause of early life 
failures. The parts screening practice was developed to remove 
such flaws. Equipment screening performed during the 1960’s 
also attests to such recognition. In the early 1970’s, Ryerson 
used defect or flaw as a parameter in his Cost Reduction Early 
Decision Information Techniques (CREDIT). In 1981, Quart 
presented some data and developed an equation relating failures 
resulting from screening to flaws (ref. 1-5), and later Wong 
extended the flaw theory to cover failures occurring during 
the normal operating period of the system (ref. 1-6). In 
essence, the combination of flaws and stresses cause;' most 
failures. In recognition of this fact, a large amount of energy 
was exerted in developing a screening technique in the late 
I970’s. Two national meetings on environmental stress 
screening of electronic hardware were held in the United W <» s . 
in 1979 and 1981, under the sponsorship of the Institute of 
Environmental Sciences. The screening guidelines documents 
distributed at the 1981 meeting indicated that a number of 
systems experienced 20- to 90-percent reduction in field failure 
rate after the addition of environmental stress screening in 
manufacturing. Reliability engineers should concentrate on 
flaws and stresses and leave the basic design to the designers. 


New Direction 

The new direction in reliability engineering will be toward 
more realistic recognition of the causes and effects of failures 
from 'he system down to the microlevel. Instead of attempting 
to operate independently, reliability engineering should work 
interactively with other engineering functions. At the system 
level, critical environmental stresses must be identified and 
quantified. Design and manufacturing flaws, internal and ex- 
ternal stresses, and failure mechanisms need to be classified 
and folded into the overall quantitative model of failure charac- 
teristics. The increasing emphasis on reliability physics has 
been bringing reliability engineering back toward the under- 
standing and application of basic engineering principles. 
Although some work has been done, the different reliability 
technical areas have not been working together to provide a 
unified methodology. For example, although thermal cycling 
has been recognized as a key factor in inducing failures, MIL- 
HDBK-2;7E (ref. 1-7) does net take into account thermal 
cycling effects on failure rates. An attempt was started to bridge 
the gap between failure rate, thermal cycling, and fatigue failure 
mechanisms in 1981 (ref. 1-8). Stress/strength, wear, and 
fatigue will still be considered in this manual, but in reference 
to their effects on flaws rather than on the basic design. 

Future emphasis should shift as indicated in figure 1-1. In 
several reliability efforts the words “dynamic” or “tailored” 
were used, signifying that flaws do not stay constant. They 
are very much human related as well as affected by the 


economic environment. Therefore, what is done to control or 
eliminate flaws must be flexible. There is no point in trying 
to eliminate something that is not there. Dynamic quality 
control will receive more emphasis, as shown at the lower right 
comer of figure 1-1 and discussed in appendix B, in achieving 
reliability, since it is a task for removing flaws. Although the 
investigation of failure physics will continue, the key now lies 
in the physics of flaw failures. For visibility and ease of system 
analysis, some quantitative measure of reliability will still be 
required. The flaw theory covers both nonconstant and 
constant failure rates (ref 1-6). The mathematical tools 
develooed with the assumption of constant failure fate will no 
longer be sufficient. An analysis published in 1988 (ref. i-9) 
indicat A th?‘ the failure rate of electronic systems generally 
Jecre.' ses witf system age, with failure humps along the way 
resemJing tie track of a roller coaster. It is, therefore, 
necessary now to deal with a roller-coaster curve, rather j 

than a b o, h’.jb curve, for electronic system hazard rates. | 

Fortunately, the advent of high-speed computers enables ! 
nonstationary failure rate models to be dealt with Ity simulation j 
or Monte Carlo methods without requiring complicated closed- : 
form mathematical expressions. A new set of mathematical 
tools is expected to be developed for use with the latest 1 
reliability models (ref. 1-10). f 

Software reliability, not shown in figure 1-1, requires j 
increasing emphasis. However, software reliability is really 
a misnomer. It has an entirely different meaning from that of f 
hardware reliability. Software reliability is a measure of ; 
software design adequacy. Therefore, it is a separate topic and ; 
is discussed in chapters 7 and 8. i 

It is proposed that new boundaries be defined for reliability ) 

engineering that exclude management, applied mathematics, j 

and double-checking. Not that these functions are not | 
important. In fact, they may still be performed by reliability , 
engineers even though they are not classified as reliability 
engineering Then, let us redefine reliability engineering in 
tighter boundaries as a synthesizing function devoted to flaw 
control . Figure 1 -4 diagrams how this function interacts with 
others. Reliability engineering would act like a filter or 
synthesizer feedback loop, performing the following tasks: 

(1) Identifying flaws and stresses and ranking them for 
priority actions 

(2) Engaging the material technologists to determine the 
flaw failure mechanisms 

(3) Developing flaw control techniques and feeding such 
information back to the engineers responsible for design, 
manufacture, and support planning 

The types of output to be expected from reliability engi- 
neering are stress screening regimens, failure characteristics 
of parts and systems, effects of environmental stresses on flaws 
and failures, relationship of failure mechanisms such as 
electromigration to fla'” failures, relationship of manufacturing 
yield to product reliability, flaw detection methods such as 
automated IC chip inspection and vibration signature monitor- 







Figure 1-4.— Role of reliability engineering for the I990‘s. 


ing, and many more outputs than an engineering function 
should provide. 

As mentioned earlier, flaws in an item depend on the design, 
manufacturing processes, quality control, parts, and materials. 
Therefore, the distribution of flaws does not stay constant. 
Reliability engineering must act dynamically to provide flaw 
control information to the proper functions for action on a 
timely basis. It is important that customers recognize this fact 
and allow proper controls to be tailored to the needs of the 
time instead of demanding a one-time negotiation on what 
should be done for the total contract period. 

Concluding Remarks 

Much of the reliability effort through the years has been 
aimed at increasing independent systems engineering and 
further refining basic design approaches. Now the time has 
come to direct our attention to flaw failures. These failures 
come from interaction of stresses and flaws. We must bring 
to bear on these flaws all the engineering techniques at our 
disposal in order to eliminate them. Reliability engineers are 
entering the era of interaction. Reliability engineering and basic 
engineering must work closely to create a synergistic effect 
for achieving ever higher reliability. 
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Reliability Training 2 


1. Who has provided a large impetus toward safe and predictable products? 

A. Industry* B. Universities C. Government 

2. What brought on the reliability problem? 

A. Use of semiconductor devices 

B. Increased complexity of equipment 

C. Material shortages 

3. How does production yield relate to reliability? 

A. There is no relationship. 

B. High yield correlates with low reliability. 

C. High yield correlates with high reliability. 

4. What is the theme of this course? 

A. Nothing is learned from failures. 

B. Failures only need to be fixed. 

C. Each failure should be studied to see what can be done about it. 


-Answers are given ai the end of this manual. 
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Chapter 2 

Reliability Mathematics and Failure Physics 

Mathematics Review 


Readers should have a good working knowledge of algebra Rule 2: 

and a slight knowledge of integral and differential calculus. 

However, for those who feel rusty in these subjects the follow- 
ing review includes solved examples for every mathematical 
manipulation used in this manual. ft u j e j : 



- = e v “ v 


Rounding Data 


Notation 

The Greek symbol L (sigma) means “take the sum of,” and 
the notation 

n 

2* 

i- I 

means to take the sum of the x ( ’s from i = 1 to i = n. 

The symbol "Vr means “take the n ,h root of.r.“ The square 
root, h/x. is usually written as 'Jx without the radicand (the 2). 

The Greek symbol II (pi) means “take the product of,” and 
the notation 

n 

n 

/*■ i 

means to take the product of the x,’s from / = 1 to / = «. 

The notation*/ is referred to as a factorial; it is a shorthand 
method of writing 1 x 2 x 3 x 4 x 5 x 6 x . . . x x; or, in 
general: x/ = x(x — 1)(* — 2) . . . (1). However, 0! is defined 
to be unity. 

Manipulation of Exponential Functions 


Reliability calculations are made by using failure rate data. 
If the failure rate data base is accurate to three places, 
calculations using these data can be made to three places. Use 
should be made of the commonly accepted rule (computer’s 
rule) to round the computational results to the proper 
number of significant figures. The “Mathematics Dictionary” 
(ref. 2-1) defines rounding off as 

When the first digit dropped is less than 5, the 
preceding digit is not changed; when the first digit 
dropped is greater than 5 or 5 and some succeeding 
digit is not zero, the preceding digit is increased by 
1; when the first digit dropped is 5 and all succeeding 
digits are zero, the commonly accepted rule is to 
m \ke the preceding digit even, i.e., add 1 to it if it 
is odd, and leave it alone if it is already even. 

For example, if the reliability of a system is 0.8324, 0.8316, 
or 0.8315, it would take the form 0.832, if rounded off to 
three places. 


An exponential function is the Napierian base of the natural 
logarithms, e = 2.71828 .... raised to some power. For 
example, e 2 is an exponential function and has the value 
7.3891. This value can be calculated on most calculators. 

Rules that must be followed when manipulating these 
functions are given here. 

Rule l: 


Integration Formulas 

Only the following integration formulas are used in this 
manual; 



* ,,+l 
« + 1 
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b B+l - a" +i 
n + 1 


(1) 
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f* I* 

j e~ x dx- = -e~ h + e~ u = e~" - 


e~ h (2) 
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Examples I : 
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(3) 


TABLE 2-1.*— binomial coefficients 


\x 2 dx = 


X ' 2 + 1 J 3 


2 + 1 3 

= ( 

2 1 2 2 2 2 
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Example 2: 


Example 3: 
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£~ 3 - * 4 




4 f- 8 -e- 6 


Differential Formulas 

Only the following differentia] formulas are used in this 
manual: 


<H ax) 

dx 

d(ax n ) 

dx 


= a 


= nax"-' 


(4) 

(5) 


Examples 4: 


d{x) 

dx 

dm 

dx 


= 1 


= 4 


Examples 5: 


d(x 2 ) 


dx 

d(4r 3 ) 

dx 


— 2v 2-1 = 2x 
= (3Hr 3 -' = 12x 2 


Partial Derivatives 

This manual uses the following partial derivative formula: 


dv dfrys) 

T~ ~ “1 — * VC 
d.V| dx 


(6) 
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Example 6: 

( x » 2 ft 

v = 2ftx3ftx4ft = 24ft 3 j y = 3 ft 

(* = 4ft 

dv 

£ * VC = 12 ft 2 


Expansion of (< a + b) n 

It will be necessary to know how to transform the expression 
(a + b)" into what is called a binomial expansion. This type 
of problem is easily solved by using table 2- 1 and recalling that 

(a + by = a" + na"- l b + <llJM a n- 2 b 2 
(» - 2)(n - 1)(») 

+ a n SfrS + . . . 

, «(« ~ i)(« - 2) . . , (n - m + 1) 
m! 

X a"~ m b m + . . . + b" (7) 

Example 1: 

Expand (a + b) 4 . From table 2-1 with n — 4, 

(a + b) 4 = a 4 + 4a 3 i> + 6 a 2 b 2 + 4 ad 3 + i 4 

Failure Physics 

When most engineers think of reliability, they think of parts. 
This is understandable, since parts are the bui'ding blocks of 
products. All agree that a reliable product must have reliable 






party. But would everyone agree on what makes a part reliable? 
When asked this question, nearly all engineers would say a 
reliable part is one purchased according to a certain source 
control document and bought from an approved vendor. Unfor- 
tunately, these two qualifications are not always guarantees 
of reliability, even though we would like to think that they 
are. To illustrate, consider the following case of the qualified 
clock. 

A clock purchased according to PD 4600008 was procured 
from an approved vendor for use in the ground support equip- 
ment of a missile system and was subjected to qualification 
tests as part of the reliability program. These tests consisted 
of high- and low-temperature, mechanical shock, temperature 
shock, vibration, and humidity tests. The clocks from the then 
sole-source vendor failed two of the tests: low temperature 
and humidity. A failure analysis revealed that lubricants in 
the clock’s mechanism froze and that the seals were not 
adequate to protect the mechanism from humidity. A second 
approved vendor was selected. His clocks failed the high- 
temperature test. In the process the dial hands and numerals 
turned black, making readings impossible from a distance of 
2 feet. A third approved vendor’s clocks passed all of the tests 
except mechanical shock, which cracked two of the cases. 
Ironically, the fourth approved vendor’s clocks, though less 
expensive, passed all the tests. 

The point of this illustration is that four clocks, each 
designed to the same specification and procured from a 
qualified vendor, all performed differently in the same 
environments. These various failures are shown in table 2-2. 
Why did this happen? The answer is simple. The specification 
did not include the gear lubricant or the type of coating on 
the hands and numerals or the type of case material. 

Many similar examples could be cited, ranging from 
requirements for glue and paint to complete assemblies and 
systems, and the key to answering these problems can best 
be stated as follows: To know how reliable a product is or 
how to design a reliable product, you must know how many 
ways its parts can fail and the types and magnitude of stresses 
that cause such failures. Think about this for a while; if you 
knew every conceivable way a missile could fail, and if you 
knew the type and level of stress required to produce each 


of these failures, you could build a missile that would never 
fail. You could do this because you could 

(1) Eliminate as many ways of failure as possible 

(2) Eliminate as many stresses as possible 

(3) Eliminate the remaining potential failures by controlling 
the level of the remaining stresses 

Sound simple? Well, it would be simple, except for one thing. 
Despite the thousands of failures observed in industry each 
day, we still knew very little about why things fail and even 
less about how to control these failures. The situation is not 
hopeless, however. Through systematic data accumulation and 
study, we learn more each day. This manual is a small but 
important part of this systematic development. 

As pointed out earlier, this manual introduces some basic 
concepts of failure physics. These include failure modes (how 
failures are revealed); failure mechanisms (what produces the 
failure mode); and failure stresses (what activates the failure 
mechanisms). It also introduces the theory and the practical 
tools available for controlling failures. 

This chapter presents some basic probability theorems in 
preparation for a discussion of the various classes of failures 
that contribute to product unreliability. 

Probability Theory 

Fundamentals 

Because reliability values are probabilities, every student 
of reliability disciplines should know the fundamentals of 
probability theory. Probability theory is used in chapter 3 to 
develop models that represent exactly how failures occur in 
products 

Probability defined.— Probability can be defined as follows: 
If an event can occur in A different ways , all of which are 
considered equally likely , and if a certain number B of these 
events are considered successful or favorable , the ratio B/A 
is called the probability of the event . Probability by this 
definition is also called an a priori (beforehand) probability 
because its value is determined without experimentation. It 
follows that reliability predictions of the success of missile 


TABLE 2-2.— RESULTS OF QUALIFICATION TESTS ON 
SOURCE CONTROL DOCUMENT CLOCK 


Vendor 

High 

temperature 

Low 

temperature 

Mechanical 

shock 

Temperature 

shock 

Vibration 

Humidity 

I 


Fail 
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flights which are made before the flights occur are a priori 
reliabilities. In other words, a priori reliabilities are estimates 
of what may happen, not observed facts. 

After an experiment has been conducted, an a posteriori 
probability or an observed reliability can be defined as follows: 
Iff( n ) is the number of favorable or successful events observed 
in a total number of n trials or attempts , the relative frequency 
f(n)/n is called the statistical probability , the a posteriori 
probability , the empirical probability, or the observed 
reliability . Note that the number of favorable events /(/i) is 
a function of the total number of trials or attempts n. Therefore, 
as the number of trials or attempts changes, /(n) may also 
change, and consequently the statistical probability (or 
observed reliability) may change. 

Reliability of a coin . —Trying out this theory, consider the 
physics of a coin. Assume it has two sides, is thin, and is made 
of homogeneous material. If the coin is tossed, one of two 
possible events may occur: heads or tails. If landing heads up 
is considered more favorable than landing taiir. up, a prediction 
of success can be made by using the a priori theory. From 
the a priori definition, the probability of success is calculated as 


1 favorable event 

2 possible events 


= 1/2, or 50 percent 


This is an estimate of what should be observed if the coin is 
tossed, but not yet an observed fact. After the coin is tossed, 
however, the probability of success could be much more 
specific as shown in table 2-3. 


TABLE 2-3. -OBSERVED PROBABILITY OF SUCCESS 


Number of tosses, n 
Number of heads 

l 

10 

100 

1000 

10 000 

observed, /(«) 
Relative frequency 

0 

7 

55 

464 

5080 

of probability of 
success, f{n)ln 

0 

0.70 
1 

0.55 

0.464 

0.508 


The table shows two important phenomena: 

(1) As the number of trials changes, the number of favorable 
events observed also changes. An observed probability of 
success (or observed reliability) may also change with each 
additional trial. 

(2) If the assumptions made in calculating the a priori 
probability (reliability prediction) are correct, the a posteriori 
(observed) probability will approach the predicted probability 
as the number of trials increases. Mathematically, the relative 
frequency f(n)/n approaches the a priori probability B/A as 
the number of trials n increases, or 

lim f(n) B 
"- 00 n ~A 


In the coin toss example, the predicted reliability was 0.50. 
The observed reliability of 0.508 indicates that the initial assump- 
tions about the physics of the coin were probably correct. If, 
as a result of 10 000 tosses, heads turned up 90 percent of 
the time, this could indicate that the coin was incorrectly 
assumed to be homogeneous and that, in fact, it was “loaded.” 
Inconsistency in the actual act of tossing the coin, a variable 
that was not considered in the initial assumptions, could also 
be indicated. Here again, even with a simple coin problem, 
it is necessary to consider all the ways the coin may “fail” 
in order to predict confidently how it will perform. 

Reliability of missiles,— In the aerospace industry a priori 
probabilities (reliability predictions) are calculated for missiles 
in an effort to estimate the probability of flight success. 
Inherent in the estimate are many assumptions based on the 
physics of the missile, such as the number of its critical parts, 
its response to environments, and its trajectory. As in the coin 
problem the ultimate test of the missile’s reliability prediction 
is whether or not the prediction agrees with later observations. 

If during flight tests the observations do not approach the 
predictions as the number of flights increases, the initial 
assumptions must be evaluated and corrected. An alternative 
approach is to modify the missile to match the initial assump- 
tions. This approach is usually pursued when the reliability 
prediction represents a level of success stated by the customer 
or when the predicted value is mandatory for the missile to 
be effective. This subject of reliability predictions is discussed 
again in chapter 4. 

In practice, reliability testing yields the knowledge needed 
to verify and improve initial assumptions. As experience is 
gained, the assumptions undergo refinements that make it 
possible to develop more accurate reliability predictions on 
new missiles and systems not yet tested or operated. This 
information also provides design engineers and management 
with data to guide design decisions toward maximum missile 
or system reliability. Some reliability problems require the 
use of Bayes or Markovian probability theorems. Additional 
information on other topics is available in references 2-2 to 
2-5 and in IEEE Reliability Society publications and other 
documents listed in the reference sections for chapters 3 to 
9 and in the bibliography at the end of this manual. 

Probability Theorems 

The three probability theorems presented here are 
fundamental and easy to understand. In these theorems and 
examples the probability of success (reliability) is represented 
with an R and the probability of failure (unreliability) with 
a Q, The following section (Concept of Reliability) examines 
what contributes to the reliability and unreliability of products. 

Theorem 7,— If the probability of success is /?, the 
probability of failure Q is equal to 1 - R. In other words, the 
probability that all possible events will occur is Q + R = 1 . 

Example 1 : If the probability of a missile flight success is 
0.81, the probability of flight failure is 1 -0.81 =0.19. 





Therefore, the probability that the flight will succeed or fail 
is 0.19 + 0.81 = 1.0. 

Theorem 2.— If /?, is the probability that a first event will 
iccur and /?» is the probability that a second independent 
event will occur, the probability that both events will occur 
is R\Ri. A similar statement can be made for more than two 
independent events. 

Example 2: If the probability of completing one countdown 
l without a failure R, is 0.9, the probability of completing two 

j; ; countdowns without failure is R,R 2 = (0.9)(0.9) = 0.81. The 

i probability that at least one of the two countdowns will fail 

' s * -#|/? 2 = 1 -0.81 =0.19 (from theorem 1). We say 
' lhat at least one will fail because the unreliability term Q 

includes all possible failure modes, which in this case is two: 
one or both countdowns fail. 

f Example 3 • If the probability of failure (?, during one 

, countdown is 0.1, the probability of failure during two 

Y countdowns is Q,Q 2 = (0.1)(0.1) = 0.01. Therefore, the 

r probability that at least one countdown will succeed is 

- 1 ~ Q1Q2 = 1 “ 0.01 = 0.99. We say that at least one will 

i. succeed because the value 0.99 includes the probability of one 

• countdown succeeding and the probability of both countdowns 

l succeeding. 

r Example 4: If the probability of completing one countdown 

without failure /?, is 0.9 and the probability of a second 
countdown failing is Q, = 0. 1 , the probability that the first 
I. wil1 succeed and the second fail is R,Q 2 = (0.9)(0. 1) = 0.09. 

[ Theorem 3. -If the probability that one event will occur is 

L and the probability that a second event will occur is R 2 and 

; if not more than one of the events can occur (i.e.. the events 

' are mutually exclusive), the probability that either the first or 

second event, not both, will occur is /?, + R 2 . A similar 
theorem can be stated for more than two events. 

I Example 5 (true event method ): Consider now the proba- 

bility of completing two countdowns without a failure. Let the 
probabilities of success for the first and second countdowns 
be R\ and R 2 and the probabilities of failure be Q\ and Q>. 
In order to solve the problem using theorem 3, it is best to 
i diagram the possible events as shown in figure 2-1. The 

mutually exclusive events are 



Total 

possible 

events 


First 

countdown 

Succeeds (fl,) 

Second 

Succeeds (ft 2 ) 


countdown 



Pails (Q 2 ) 


Falls (O,) 




« 1 0 2 
a 1 


Figure 2-1.— Diagram of possible events— probability of completing two 
countdowns without a failure. 



Q 1 first countdown fails 

R 1 Q 2 first countdown succeeds and second fails 

R\R 2 both countdowns succeed 

From theorem 3 the probability that one of the three events 
will occur is 


Q\ + R\Qi R\R 2 

But because these three events represent all possible events 
that can occur, their sum equals I (from theorem 1). Therefore, 

Qi + R\Q 2 + R\R 2 — 1 

The probability of completing both countdowns without one 
failure R t R 2 is the solution to the proposed problem; 
therefore, 

R { R 2 = I - (R,Q 2 + £,) 

If /?| = 0.9, Gi = 0 . 1 , /?,= 0.9, and Q 2 = 0. 1 then 

RiR 2 = 1 - [(0.9)(0. 1) + 0. IJ 

= 1 - (0.09 + 0.1) = 1 - 0.19 = 0.81 

which agrees with the answer found in example 2 by using 
theorem 2. The expression for R,R 2 can also be written 

/?i/? 2 = 1 - (R t Q 2 + g|) = 1 - ((I - Q t )Q : + £|] 

= I ~(Qi + Q 2 ~Q,Q 2 ) 

i 

which is the usual form given for the probability of both events \ 

succeeding. Note, however, that in this expression, the event 1 

indicated by G 1 G 2 (both countdowns fail) is not a true pos- ] 

sible event, because we stipulated in the problem that only one 1 

countdown could fail. The term Q\Q 2 is only a mathematical ■ 

event with no relation to observable events. In other words, ! 

if the first countdown fails, we have lost our game with chance. ! 

Example 6 ( mathematical event method): Now consider the ! 

same problem as in example 5, ignoring for the time being 
the restriction on the number of failures allowed. In this case 
the diagram of the possible events looks like that shown ir 
figure 2-2, In this case the mutually exclusive events are 
i?i /?2 both countdowns succeed 
R\Q 2 first countdown succeeds and second fails 
Q\R 2 first countdown fails and second succeeds I 

Q\Q 2 both countdowns fail i 

Keep in mind that in this example both countdowns may fail. 

From theorem 3 the probability that one of the four events ! 

will occur is | 


*1*2 + + Q\R 2 + <?,& 
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Total 

possible 

events 


First 

Succeeds ( R ^ 

Second 

countdown 


countdown 





Fails (O^ 


Succeeds ( R 2 ) 


Fails (Q 2) 


Second 

countdown 


Succeeds R* 


Fails (Q 2 ) 


r \ R 2 


/? 1 02 

Q^2 

0^2 


Figure 2 - 2 .— Diagram of possible events— number of failures not restricted. 


Again, because the four events represent all possible events 
that can occur, their sum equals unity (from theorem 1); that is, 

RxR 2 + R\Q2 + Q\R 2 + Q\Q2 = 1 

Solving for the probability that both countdowns will succeed i.; 


R\Ri — 1 (/?i02 “*■ Q\Ri + Q\Qi) 


Substituting 1 - Q } for R\ and i - Q 2 for R 2 on the righ*. side 
of the equation gives the answer given in example 5: 


= l - td - Qx)Q 2 + 0,d - Qi) + QiQzi 
- 1 - (g 2 ~ QiQz + Q\ ~ O 1 O 2 + Q\Qi) 
= 1 ~ (0. + 0 2 - QxQz) 


This countdown problem has been solved in two ways to 
acquaint you with both the true event method and the mathe- 
matical event method of determining probability diagrams. The 
exercises at the end of this chapter may be solved by using 
whichever method you prefer. Because these exercises will 
be helpful to you in gaining a working knowledge of the three 
theorems presented, we suggest that you work the problems 
before continuing to the next section. 


Concept of Reliability 

Now that you have an understanding of the concepts of 
probability a .cl failure physics, you are ready to consider the 
concept of reliability. First, the most common definition of 
reliability— in terms of successful operation of a device— is 
discussed. That definition, to fit the general theme of this 
manual, is then modified to consider reliability in terms of 
the absence of failure modes. 




Reliability as Probability of Success 

The classical definition of reliability is generally expressed 
as follows: Reliability is the probability that a device will 
operate successfully for a specified period of time and under 
specified conditions when used in the manner and for the 
purpose intended. This definition has many implications. The 
first is that when we say that reliability is a probability, we 
mean that reliability is a variable, not an absolute value. 
Therefore, if a device is 90 percent reliable, there is a 10 
percent chance that it will fail. And because the failure is a 
chance, it may or may not occur. As in the coin example, as 
more and more of the devices are tested or operated, the ratio 
of total success to total attempts should approach the stated 
reliability of 90 percent. The next implication concerns 
the statement . . will operate successfully . . This 
means that failures that keep the device from performing its 
intended mission will not occur. From this comes a more 
general definition of reliability: that it is the probability 
of success. 

It should be obvious then that a definition of what constitutes 
the success of a device or a system is necessary before a 
statement of its reliability is possible. One definition of success 
for a missile flight might be that the missile leaves the 
launching pad. Another, that the missile hits the target. Either 
way, a probability of success, or reliability, can be determined, 
but it will not be the same for each definition of success. The 
importance of defining success cannot be overemphasized. 
Without it a contractor and a customer will never reach an 
agreement on whether or not a device has met its reliability 
requirements (i.e., the mission). 

The latter part of the classical definition indicates that a 
definition of success must specify the operating time, the 
operating conditions, and the intended use. Operating time is 
defined as the time period in which the device is expected to 
meet its reliability requirements. The time period may be 
expressed in seconds, minutes, hours, years, or any other unit 
of time. Operating conditions are defined as the environment 
in which the device is expected to operate; they specify the 
electrical, mechanical, and environmental levels of operation 
and their durations. Intended use is defined as the purpose of 
the device and the manner in which it will be used. For 
example, a missile designed to hit targets 1000 miles away 
should not be considered unreliable if it fails to hit targets 1 100 
miles away. Similarly, a set of ground checkout equipment 
designed to be 90 percent reliable for a 1-hour tactical 
countdown should not be considered unreliable if it fails during 
10 consecutive countdowns or training exercises. The proba- 
bility of success in this case is (0.9) 10 - 0.35 (from probability 
theorem 2). 

In addition to these specified require, rents, we must also 
consider other factors. As explained in the inherent product 
reliability section of this chapter, these areas have a marked 
effect on the reliability of any device. 
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Reliability as Absence of Failure 

Although the classical definition of reliability is adequate 
for most purposes, we are going to modify it somewhat and 
examine reliability from a slightly different viewpoint. 
Consider this definition: Reliability is the probability that the 
critical failure modes of a device will not occur during a 
specified period of time and under specified conditions when 
used in the manner and for the purpose intended. Essentially, 
this modification replaces the words “a device will operate 
successfully” with the words “critical failure modes . . . will 
not occur.” This means that if all the possible failure modes 
of a device (ways the device can fail) and their probabilities 
of occurrence are known, the probability of success (or the 
reliability of a device) can be stated. It can be stated in terms 
of the probability that those failure modes critical to the per- 
formance of the device will not occur. Just as we needed a 
clear definition of success when using the classical definition, 
we must also have a clear definition of failure when using the 
modified definition. 

As an example, assume that a resistor has only two failure 
modes: it can open or it can short. If the probability that the 
resistor will not short is 0.99 and the probability that it will 
not open is 0.9, the reliability of the resistor (or the probability 
that the resistor will not short or open) is given by 

^resistor = Probability of no opens x Probability of no shorts 
= 0.9 X 0.99 = 0.89 

Note that we have multiplied the probabilities. Probability 
theorem 2 therefore requires that the open-failure-mode 
probability and the short-failure-mode probability be independ- 
ent of each other. This condition is satisfied because an open 
failure mode cannot occur simultaneously with a short mode. 

Product Application 

This section relates reliability (or the probability of success) 
to product failures. 

Product failure modes .— In general, critical equipment 
failures may be classified as catastrophic part failures, 
tolerance failures, and wearout failures. The expression for 
reliability then becomes 

R = PcP,K 

where 

P c probability that catastrophic part failures will not occur 
P, probability that tolerance failures will not occur 
P w probability that wearout failures will not occur 
As in the resistor example these probabilities are multiplied 
together. This means they are considered to be independent 
of each other, but this may not always be true because an out- 
of-tolerance failure, for example, may evolve into or result 
from a catastrophic part failure. Nevertheless, in this manual 
they are considered independent and exceptions are pointed 
out as required. 


Inherent product reliability .— The next step is to consider 
the inherent reliability of a product. Try to think of the 
expression P t P,P w as representing the potential reliability of 
a product as described by the product’s documentation. Or 
to put it another way, let it represent the reliability inherent 
in the design drawings instead of the reliability of the manufac- 
tured hardware. This inherent reliability is predicated upon 
the decisions and actions of many people. If they should 
change, the inherent reliability could change. 

If the inherent reliability of the design is denoted by /?,, then 

R, = W„ 

Why do we consider inherent reliability? Because the facts of 
failure are these: When a design comes off the drawing board, 
the parts and materials have beei selected; the tolerance, error, 
stress, and other performance analyses have been performed; 
the type of packaging is firm; the manufacturing processes and 
fabrication techniques have been decided; and usually the test 
methods and the quality acceptance criteria have been selected. 
At this point the design documentation represents some potential 
reliability that can never be increased except by a design change 
or good maintenance. However, the possibility exists that the 
actual reliability observed when the documentation is trans- 
formed into hardware will be much less than the potential 
reiiability of the design. To understand why this is true, 
consider the hardware as a black box with a hole in both the 
top and the bottom. Inside the box are potential failures that 
limit the inherent reliability of the design. When the hardware 
is operated, these potential failures fall out the bottom (i.e., 
operating failures are observed). The rate at which the failures 
fall out depends on how the box or hardware is operated. 
Unfortunately, we never have just the inherent failures to 
worry about because other types of failures are being added 
to ti:? box through the hole in the top. These other failures 
are generated by the manufacturing, quality, and logistics 
functions, by the user or customer, and even by the reliability 
organization itself. We discuss these added failures and their 
contributors in the following paragraphs but it is important 
to understand that, because of the added failures, the observed 
failures will be greater than the inherent failures of the design. 

K Factors 

The other contributors to product failure previously mentioned 
are called K factors; they have a value between 0 and I, and 
modify the inherent reliability as follows: 

^product = 

where 

K q probability that quality test methods and acceptance 
criteria will not degrade the inherent reliability. An 
example of K q is the situation in which the quality control 
engineer accepts a defective part that later shows up as 
a field failure and is counted against product reliability. 
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K„, probability that manufacturing processes and fabrication 
and assembly techniques will not degrade the inherent 
reliability. Examples of K m would be cold-soldered 
joints, poor lamination of multilayer printed circuit 
boards, and loose fittings in plumbing installations that 
can show up as field failures. 

K r probability that activities performed by the reliability 
engineer will not degrade the inherent reliability. An 
example of K, would be an inaccurate test analysis that 
forces a design change which degrades rather than 
improves the hardware performance. 

Ki probability that logistics activities will not degrade the 
|nherent reliability. An example of K, would be an 
inaccurate procedure in a repair manual that, if followed, 
would create more failures than it fixes. 

K„ probability that the user or customer will not degrade the 
inherent reliability. Examples of K„ are operator errors 
that cause a field failure because correct operating 
procedures are not followed. This factor has been 
observed to be quite large for many systems. In one missile 
system, II out of every 100 countdowns were aborted 
because of operator errors (i.e., K u = 0.89). 

There are many other K factors, but these are the main ones. 
Even if each K factor could be made equal to unity (which, 
of course, is the goal), we would still be left with R„ the 
inherent reliability of the design. It is also clear that any one 
of the factors can cause the product reliability to go to zero. 
The achievement of inherent reliability during production of 


a product and the achievement of reliability growth during the 
build, use, and test phases are of major concern to many 
reliability engineers. 


Concluding Remarks 

Chapter 2 has explained two principal concepts: 

(1) To design a reliable product or to improve a product, 
you must understand first how the product can fail and then 
how to control the occurrence of those failures. 

(2) There is an upper limit on how reliable a product can 
be when a certain traditional way of design and fabrication 
is used. That limit is the inherent reliability. Therefore, the 
most effective reliability engineer is the designer because all 
tf e designer’s decisions directly affect the product’s reliability. 
The three probability theorems were also illustrated in this 
chapter. 
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Reliability Training 1 

la. What notation means to take the sum of the x,'s from / = 1 to / = „? 

A. Lx’s B. Ex k C. Ex, 

'=1 /=! 

lb. Ifx= 100, = 90, * 2 - 70, and jr 3 = 50, what is E (x-x,) 2 ? 

i = i 

A. 350 B. 35X10 2 C. 35 000 

2a. What notation means to take the n ,h root of x? 

A. x n B. e" C. n € 

2b. If * = 100, x, = 90,x 2 = 70, and = 50, what is \l E (* - x,) 2 ? 

▼ ,= i 

A. 3.6 B. 59.2 C. 640 

3a. What notation means to «ike the product of the jr, 's from i = 1 to /?? 

a. n x's b. n Xk c. n Xi 

i=0 1=1 

3 

3b. If x, = 0.9, x 2 = 0.99, and x 3 = 0.999, what is II x, ? 

i = i 

A. 0.890 B. 0.800 C. 0.991 

4a. The notation xl refers to what shorthand method of writing? 

A. Poles B. Factorial C. Polynomials 
4b. What does 101/8! equal? 

A. 800 B. 900 C. 90 

5a. Describe the three rules for manipulation of exponential functions. 

i. Products 

A. Substract exponents B. Add exponents C. Multiply exponents 

ii. Negative exponent 

A. Cancel exponents B. Balance exponents C. I /Exponent 

iii. Division 

A. Add exponents B. Subtract exponents C. Multiply exponents 
5b. Simplify, «V/« 4 . 

A. t 2 B. t 4 C. « J 


6. What is the integral of the following functions? 
a. rV<fr 

JjT, 

A. or 4 /4 B. C. [U 2 ) 4 - U,) 4 J/4 

'Answers ere given et the end of this manual. 
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[ -ax . -/it. - ) i-f' 1 

t ‘-e 2 ]/fl C. — e ~ a '/a j 

7. What is the derivative of the following functions? 
a. IQ* 4 

A. 4Qr B. 4Qx 3 C. IQ* 3 
b. t 2 * 

A. e 1 ' B. f^/2 C. 2f 2 ' 

8a. Write the first two terms of the binomial expansion (a + b) n . ' 

A. a" + (n - l)a n ~'b + . . . B . a" - na"~'b + . . . C . a n + na"~'b + . . . ! 

! 

8b. Expand (a + b ) 3 by using table 2-L 

A. a 3 4- 2u 2 b + ft 3 B. a 3 - 3a 2 b — 3ab 2 + b 3 C. a 3 + 3 a 2 b 4- 3ab 2 + b 3 

9. What needs to be done to design a reliable product? • 

A. Test and fix it j 

B. Know how its parts fail 

C. Know the type and magnitude of stresses that cause such failures 4 

D. Both B and C 

10. What are a priori reliabilities estimates of? j 

A. What may happen B. What will happen C. What has happened 

11. What are a posteriori reliabilities observing? 

A. What may happen B. What has happened C. What will happen 

12. If the probability of success is /?, what is the probability of failure Q? 

A. I + R B. 1 - R 2 C. 1 - R 


13. If /?|, /?2* and ^3 are the probabilities that three independent events will occur, what is the p jollity 
that all three will occur? 

3 

A. /?[ 4* /?2 4" /?3 B. /?j(/?2 4" R$) C. II Rj 

/= I 

14. If /?,, /? 2 , and /? 3 are the probabilities that three independent events will occur and not more than one 
of the events can occur, what is the probability that one of these events will occur? 


A. B. Ry{R\ 4" /?]) C. L Ri 

i - 1 j 

} 

J 

j 


■ ' ■> rpr x, r j s u^»vm 












15. What do we need to know if a device is to perform with classical reliability? 

A. Operating time and conditions 

B. How it will be used 

C. The intended purpose 

D. All of the above 

16. What do we need to know if a device is to perform with reliability defined as the absence of failure? 

A. Critical failure modes 

B. Operating time and conditions 

C. How it will be used 

D. The intended purpose 

E. All of the above 

17. What is the inherent reliability /?, of the product you are working on? 

A. P c (the probability that catastrophic part failures will not occur) 

B. P, (the probability that tolerance failures will not occur) 

C. P w (the probability that wearout failures will not occur) 

D. The product of all of the above 

18. What is the reliability of your product? 

A. K q (the probability that quality test methods will not degrade if,) 

B. K m (the probability that manufacturing processes will not degrade if,-) 

C. K, (the probability that reliability activities will not degrade Rj) 

D. K ( (the probability that logistic activities will not degrade if,) 

E. K„ (the probability that the user will not degrade /?,-) 

F. The product of all of the above and if, 
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Chapter 3 


Exponential Distribution and Reliability Models 


An expression for the inherent reliability of a product was 
given in chapter 2 as (ref. 3-1) 

*/ = W. 

where 

P c probability that catastrophic part failures will not occur 
P , probability that tolerance failures will not occur 
p « probability that wearout failures will not occur 

In chapter 3, we discuss the term P c and develop and explain 
its mathematical representation in detail. We then use the 
probability theorems to establish metnods of writing and 
solving equations for product reliability in terms of series an J 
redundant elements. 


Exponential Distribution 

To understand what is meant by exponential distribution, 
first examine a statistical function called the Poisson 
distribution. This distribution is expressed as (ref. 3-2) 


P(x,t) 


PaVe-to 

xl 


where 

X average failure rate 
/ operating time 
x observed number of failures 


This distribution states that if an observed average failure rate 
X is known for a device, it is possible to calculate the 

probability P(x,t ) of observing x = 0.1, 2,3 number of 

failures when the device is operated for any period of time f. 

To illustrate, consider a computer that has been observed 
to make 10 arithmetic errors (or catastrophic failures) for every 
hour of operation. Suppose we want to know the probability 
of observing 0, I , and 2 failures during a 0.01-hour program. 
From the data given, then 


X (failure rate) = 10 faiiures/hour 
t (ope ating time) = 0.01 hour 
x (observed failures) = 0. 1 , and 2 

The probability of observing no failures P(0. 0.01) is then 


P( 0, 0.01) - 


(10x0.01)°e- <IOxt)OI > 


1 x e ~ c •' 


0 ! 


= e~ 0 ' =0.905 


The probability of observing one failure /*(!. 0.01) is 
(10 xO.OI) 1 e -" 0x00 » 


P( 1,0.01) = 


1! 

(0.1) 1 c ~ 01 
1 


= 0.1 X 0.905 = 0.091 


The probability of observing two failures P(l, 0.01) is 


P( 2. 0.01) = 


(10 x 0.01) 2 * - < IOxUO| > 
_ _ 


(0.1) 2 e~ 0 1 _ 0.01 x 0.905 

2X1 “ 2 


0.00905 

2 


0.0045 


Remember that the definition of P c is the probability that 
no catastrophic failures will occur. So for the computer 
P, - P( 0, 0.01) = 0.905. In other words, there is a 90.5- 
pcrcent chance that no arithmetic errors will occur during the 
0.01-hour program. This is the reliability of the computer for 
that particular program. 

Again the Poisson distribution for x = 0 (i.e., no observed 
failures) is 


INTENTIONALIT BUNK 


PRECEDING PAGE BLANK NOT FILMED 
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The term e is called the exponential distribution and is the 
simplest form of P c . Consequently, for a device that has an 
average failure rate X the probability of observing no failures 
for a period of time / is (ref. 3-3) 

P c = 

The expression for inherent reliability now takes the form 


= e^P,P K 


or in the more general expression for total product reliability 
R = e-'PfwtKJCJC.Kfi.) 

At this point it is probably a good idea to digress for a 
moment to explain why these expressions for reliability may 
differ from those used elsewhere. During the conceptual and 
early research and development phases of a program, it is 
common practice (and sometimes necessary because of a lack 
of information) to assume that P, = 1 (the design is perfect), 
that P lx = 1 (no wearout failures will occur), and that the K 
factors all equal 1 (there will be no degradation of inherent 
reliability)- These assumptions reduce the inherent reliability 
and product reliability expressions to 

R, = R = e ~ x ' 


Frequently, these assumptions are not realistic and the resultant 
reliability predictions are usually high. They may bear little 
resemblance to the reliability finally observed when the product 
is tested. Later in this manual 've will let 

P,= R = e~* 
to keep the notation simple. 

On the other hand. is also common to use to represent 
the observed product reliability. In this case the observed 
average failure rate X represents the combination of all types 
of failures including catastrophic, tolerance, and wearout. If 
the total product failure rate is X'. then 

R = e - yi = e-'PfAKJCmKAK.) 

Failure Rate Definition 

The failure rate X as used in the exponential distribution 
represents random catastrophic part failures that occur 
in so short a time that they cannot be prevented by scheduled 
maintenance (ref. 3-4). Random means that the failures occur 
randomly in time (not necessarily from random causes as many 
people interpret random failure) md randomly from part to 
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port. For example, suppose a contractor uses 1 million integrated 
circuits in a computer. Over a period of time she may observe 
an average of one circuit failure every 100 operating hours. 
Even though she knows the failure rate, she cannot say which 
one of the million circuits will fail. All she knows is that, on 
the average, one will fail every 100 hours. In fact, if a failed 
circuit is replaced with a new one, the new one, theoretically, 
has the same probability of failure as any other circuit in the 
computer. In addition, if the contractor performs a failure 
analysis on each of the failed circuits, she may Find that every 
failure is caused by the same mechanism, such as poorly 
welded joints. Unless she takes some appropriate corrective 
action, she will continue to observe the same random failures 
even though she knows the failure cause. 

A catastrophic failure is an electrical open or short, a 
mechanical or structural defect, or an extreme deviation from 
an initial setting or tolerance (a 5-percent-tolerance resistor 
that deviated beyond its end-of-life tolerance, say to 20 percent, 
would be considered to have failed catastrophically). 

The latter portion of the failure rate definition refers to the 
circumstance under which a failure is revealed. If a potential 
operating failure is corrected by a maintenance function, such 
as scheduled preventive maintenance, where an out-of- 
tolerance part could be replaced, that replacement cannot be 
represented by X because it did not cause an operating or 
unscheduled failure. Here we see one of the many variables 
that affect the operating failure rate of a product: the main- 
tenance philosophy. 


Failure Rate Dimensions 


Failure rate has the dimension of failure per unit of time, 
where the time is usually expressed in 10 T hours or cycles. 
Some Government documents express X in percent failures 
per 10 3 hours. Table 3-1 shows the most common usage. 
Generally, the form that allows calculations using whole 
numbers, rather than decimal fractions, is chosen. 


‘Bathtub Curve” 


In tlie Poisson distribution, X was referred to as an average 
failure rate, indicating that X may be a function of time X(f ) . 


TABLE 3-1.— COMMON FAILURE RATE 
DIMENSIONS 


Failures/hour, 

percent 

Failures/ 
I0 6 hours 

Failures/ 
10 9 hours 

10.0 

100.0 

100 000.0 

1.0 

10.0 

10 000.0 

.1 

1.0 

1 000.0 

.01 

.1 

100.0 

.001 

.01 

10.0 

.0001 

.001 

1.0 

.00001 

.0001 

.1 

,000001 

.00001 

.01 

.0000001 

.000001 

.001 




a 
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Figure 3-1. —Failure rate curves. 


Figure 3-1 shows three general curves representing X(r) 
possibilities. Curve A shows that as operating time increases, 
failure rate also increases. This type of failure rate is found 
where wearout or age is a dominant failure mode stress (e.g., 
slipped dutches or tires). Curve B shows that as operating time 
increases, the failure rate decreases. This type of failure rate 
has been observed in some electronic parts, especially semi- 
conductors. Curve C shows that as operating time increases, 
the failure rate remains constant. This type of failure rate has 
been observed in many complex systems and subsystems. In 
a complex system (i.e., a system with a large number of parts) 
parts having decreasing failure rates reduce the effect of those 
having increasing failure rates. The net result is an observed 
near-constant failure rate for the system. Therefore, part failure 
rates are usually given as a constant, although in reality they 
may not be. This manual deals only with constant part failure 
rates because they are related to system operation. Even if the 
failure rates might be changing over a period of time, the 
constant-failure-rate approximation is used. 

If the failure rate for a typical system or complex subsystem 
is plotted against operating life, a curve such as that shown 
in figure 3-2 results. The curve is commonly referred to as 
a "bathtub*' curve. The time t 0 represents the time at which 
the system is first put together. The interval from t 0 to r, 
represents a period during which assembly errors, defective 
parts, and compatibility problems are found and corrected. 
As shown, the system failure rate decreases during this 
debugging, or burn-in, interval as these gross errors are 
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Figure 3-2.— Failure rale versus operating time. 



eliminated. The interval from t, to h represents the useful 
operating life of the equipment and is generally considered 
to have a constant failure rate. It is during this time that the 
expression P,. = e~^ is used. Therefore, when using e ->J , 
we assume that the system has been properly debugged. In 
practice this assumption may not be true, but we may still 
obtain an adequate picture of the expected operating reliability 
by accepting the assumption. The interval from t 2 to i 3 
represents the wearout period, during which age and de- 
terioration cause the failure rate to increase and render the 
system inoperative or extremely inefficient and costly to 
maintain. 

The following analogy should help summarize the concepts 
of failure and failure rate: A company picnic is planned to 
be held on the edge of a high cliff. Because families v.iil be 
invited, there will be various types of people involved: large, 
small, young, and old, each type with its own personality and 
problems. Picnic officials are worried about the possibility of 
someone falling over the cliff. The question is. What can be 
done about it? Four possible solutions are presented: 

(1) Move the picnic farther back from the cliff. The farther 
back the picnic, the less the chance that someone will walk 
as far as the cliff and fall over. 

(2) Keep the picnic short. The shorter the picnic, the less 
time anyone has to walk to the cliff. 

(3) Look over the cliff to see if anyone has fallen. This is 
a good idea because they would know when to call the 
ambulance— but it hardly helps to keep others from falling. 
It is possible, however, that if they go to the bottom of the 
cliff to see who has fallen over, they might observe that every 
15 minutes one person over the age of 99 falls over the cliff. 
Knowing this, all persons over 99 could be sent home and the 
picnic could be saved from further tragedy. 

(4) Finally, they could build a high fence to separate the 
cliff from the picnic. Obviously, this is the best solution, 
because it is doubtful that anyone would climb the fence just 
to get to the cliff. 

Now, let us look at the analogy of this picnic-to-failure rate. 
Say that we are building a system (picnic) made of many parts 
(people) and that there are many types of parts; some are large, 
some small, and some new and untried, such as integrated 
circuits. Some of these parts, the composition resistors for 
instance, are old and mature. Each part has its own personality 
(the way it was fabricated). Our problem is how to keep these 
parts from failing (falling over the cliff). And again we have 
four possible solutions: 

(1) Reduce the stresses on the parts (move the picnic back 
from the cliff); the lower the stresses, the fewer the failures. 

(2) Keep the operating time (the picnic) short; the shorter 
the operating time, the less chance a part has to fail. 

(3) Establish part failure rates (look over the cliff to see 
if anyone has fallen), but this only helps if we know what parts 
(people) are failing. Once we know this, we can eliminate those 
parts from our system. 
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(4) Eliminate the failure mechanisms of the part (build a 
fence to separate the cliff from the picnic). This is the best 
answer, of course, because if we eliminate the cause of part 
failures, we cannot have any system failures. 


Mean Time Between Failures 


For the exponential distribution the reciprocal of failure rate 
is called the mean time between failures (MTBF) and is the 
integral of the exponential distribution: 


1 r 

MTBF = - 
X 1 


o 





Therefore, if a device has a failure rate of one failure per 100 
hours its MTBF is 100 hours. 

If the time dimension is given in cycles, the MTBF becomes 
mean cycles between failures (MCBF), a term also in common 
use. For a nonrepairable device, mean time to failure (MTTF) 
is used instead of MTBF. For a repairable device MTBF is 
usually equal to MTTF. 

If a device has an MTBF of, for example, 200 hours, this 
does not mean that the device will not fail until 200 operating 
hours have accumulated, nor does it mean that the device will 
fail automatically at 200 hours. MTBF is exactly what is says: 
a mean or average value. This can be seen from 


.-//MTBF 


When the operating time / equals the MTBF, the probability 
of no failure is 


^MTBF/MTBF =r -l =0J68 


(using exponential tables or a slide rule), which means that 
there is a 1 - 0.368 = 0.632 chance that the device will fail 
before its MTBF is reached. In other words, if a device has 
an MTBF of 1000 hours, replacing the device after 999 hours 
of operation will not improve reliability. To show the concept 
of a mean value in another way, consider the following 
empirical definition of MTBF: 


MTBF = 


Total test hours 


Total observed failures 


For example, if 100 transistors are tested for 1000 hours each 
and five failures arc observed, the observed MTBF is 


100 x 1000 100 000 

MTBF - — — — = — — = 20 000 hours 
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Note that when the failures were observed is not indicated. 
The assumption of a constant failure rate leads to a constant 
time between failures, or MTBF. 


Calculations of P c for Single Devices 
If a failure rate for a device is known, the probability of 


observing no failures for any operating period / can be 
calculated. 

Example 1 : A control computer in a missile has a failure 
rate of I per 10 2 hours. Find P c for a flight time of 0.1 hour. 
Solution /: 


p = e - 1 x '0 ^ _ <,-0.001 -0.999 


Therefore, there is one chance in a thousand that the control 
computer will fail. (Note: if X/ or //MTBF is less than 0.01, 
P c s I - X/, or 1 - //MTBF.) For example. 


p _ <,-o.ooi ^ j _ o.OOl = 0.999 


If X/, or //MTBF, is greater than 0.01 , use exponential tables 
to find P n as shown here. 




P c = e 


-- 0.923 


Example 2: The same type of problem can be solved if the 
MTBF is known. The MTBF of a tape reader used in ground 
support equipment is 100 hours. Find P c for a 2-hour 
operation. 

Solution 2: 


p — ^ — f/MTBF _ <,-2/100 __ <,-0.02 _ Q 9gQ 


If a specific P ( is required for a specified operating time, the 
required failure rate, or MTBF, can be calculated. 

Example 3: A relay is required to have a 0.999 probability 
of not failing for 10 000 cycles. Find the required failure rate 
and MCBF. 

Solution 3: 


R = e~ u 

0 999 = 0 -0,001 _ ^-Xdir 4 cycles* 


Equating exponents gives 


X(10 4 cycles) = 0.001 

0.001 I failure 


X = 


I0 4 I0 7 cycles 


-r'"' '-“v v 1 
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The required MCBF is therefore 

MCBF = ^ = 10 7 cycles 


Example 4: A system has 100 parts, each one required for 
system success. Find the system reliability R x if each part has 
R = 0.99. 

Solution 4 : 


Reliability Models 

In the following sections we replace P l . = e~ Kl , the 
reliability of a part, with a plain R to keep the notation simple. 

Calculation of Reliability for Series-Connected Devices 

In reliability, devices are considered to be in series if each 
device is required to operate without failure to obtain system 
success (ref. 3-5). A system composed of two parts is 
represented in a reliability diagram, or model, as shown in 
figure 3-3. If the reliability R for each part is known, from 
probability theorem 2, chapter 2, the probability that the 
system will not fail is 

R s = R\R 2 

(We assume that the part reliabilities are independent; i.e., 
the success or failure of one part will not affect the success 
or failure of another part.) If there are n parts in the system, 
each one required for system success, the total system reli- 
ability is given by 

« 

R s = R\RiR^ . . . R n — IT Rj 

/— i 


where 

R s probability that system will not fail 
Rj reliability of / h part 
n total number of parts 

The expression 

n 

r s = n ^ 

j = i 


is often called the product rule. 

• >: 



Figure 3-3.— Series model. 



n 100 

R s — n Rj ~~ 11 Rj — R]RiRy . . . R\qo 

j = i J = i 


= (0.99)(0.99)(0.99) , . . (0.99) = (0.99) 100 


= (e~° oijloo = ,-. =0368 


Theretore, the probability that the system will succeed is about 
37 percent. 

Example 5: For a typical missile that has 7000 active parts 
and a reliability requirement of 0.90, each part would have 
to have a reliability R p of 0.999985. This is calculated from 

( /?,, ) 7000 - 0.90 = £ -0 105 
Solution 5: Therefore, 

R = (^-0.105)1/7000 _ 1.5x10“* _ ^-0.000015 

= 1 -0.000015 =0.999985 
The product rule can also be expressed as 


n 

R s — II Rj — R[R-fR^ . . . R n 
7=1 

— e ~ h l : P ~ \, { t , 

— £# “IVl + + • V»l 

= exp 

where 

A, failure rate of / h part 
tj operating time of y th part 

Therefore, if for each series-connected part in a system me 

failure rate and operating time are known, the system reliability 

n 

can be calculated by finding E Ay/, and raising e to the 
/ " \ i=l 

- [ L A fj power. 

\» / 
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Figure 3-4 — Series mode! Using failure rales and operating limes. 


Example 6: Find the system reliability from the model 
shown in Figure 3-4. 

Solution 6: Step i 

3 

Et \jtj = Xj/| + \2^2 4* 

7=1 

- I0/10 5 (10) + 20/10 3 (4) + 100/10 3 (2) 

= 100/I0 3 4 80/ 10 3 + 200/10 3 = 380/I0 3 

Step 2 



If the t/s are equal (i.e., each part of the device operates for 
the same length of time), the product rule can further be 
reduced to 


Calculation of Reliability for Devices Connected in Parallel 
(Redundancy) 

In reliability, devices are considered to be in parallel if one 
or more of the devices can fail without causing system failure 
but at least one of the devices must succeed for the system 
to succeed. First we consider simple redundancy. 

Simple redundancy .— If n devices are in parallel so that only 
one of the devices must succeed for the system to succeed, 
the devices are said to be in simple redundancy. The diagram, 
or model, of a two-part redundancy system presented in figure 
3-6 illustrates this concept. In other words, if part 1 fails, the 
system can still succeed if part 2 does not fail, and vice versa. 
However, if both parts fail, the system fails. 

From probability theorem 3, chapter 2, we know that the 
possible combinations of success R and failure Q of two devices 
is given by 

+ #102 + Q\&2 + Q\Ql 

where 



where t c is the common operating time. 

Example 7: Find the reliability of the system shown in 
figure 3-5. 

Solution 7: Step 1 


R]R 2 both parts succeed 
R\Q 2 part 1 succeeds and part 2 fails 
Qi /?2 part 1 fails and part 2 succeeds 
Q { 2: both parts fail 

We also know that ihe sum of these events equals unity, since 
they are mutually exclusive (i.e. , if one event occurs the others 
cannot occur). Therefore, 


E Xj , - X, + X 2 + X 3 = 7/10 3 + 5/10 3 + 6/10 3 = 18/10 3 
7=1 

Step 2 



/?|/?2 + R\Q2 + Ql^2 4 Q\Qz - 1 

Because at least one of the parts or devices must succeed in simple 
redundancy, the probability of this happening is given by 

R\R 2 4* RyQ 2 4 Q\Ri - 1 - Q1Q2 

In simple terms, if the only way the redundant system can fail 
is by all redundant parts failing, the probability of success must 
be equal to 1 minus the probability that all redundant parts 



Figure 3-5.— Series model with operating times equal. 
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Figure 3-6 — Simple redundancy model. 


will fail (i.e., R = 1 - Q), from probability theorem 1, 
chapter 2. This reasoning can be extended to n redundant parts 
if at least one of the n parts must succeed for the system to 
succeed. 

Example 8: Suppose there are three ways that a space 
capsule can be guided: (1) automatically with /?, =0.9, 
(2) semiautomatically with R 2 = 0.8, (3) manually with 
/? 3 = 0.7. The model or diagram of successful guiding, 
assuming that the three ways are independent of each other, 
is shown in figure 3-7. From probability theorem 3, chapter 2, 
the possible events are given by 

R\R 2 Rz + R\R 2 Qz + R\Q 2 Rz + Q\E 2 Rz + ^10203 

+ 0102^3 + 01^203 + 010203 

Because the sum of these probabilities is equal to unity and 
at least one of the control systems must operate successfully, 
the probability that guidance will be successful /? gui(Jancc » 

^guidance ~ R\R 2 Ri + R\R 2 Qi + R\Q 2 Ri + Q\R 2 Ry 

+ R\QiQi + Q\QiR* + QfaQi 

= l - Q\Q 2 <h = l - [(l - /?,)(! - R 2 )(\ - /?,)] 

= I - [(I - 0.9)(1 - 0.8)(1 - 0.7)] 

= 1 -[(0.1)(0.2)(0.3)] 

= 1 - (0.006) - 0.994 

In general, then, for simple redundancy 

n 

^simple redundant ** 1 — H Qj— 1 — (Q\Q 2 Qj . . . 6n) 

j= I 

where 

n 

II Qj total probability of failure 
i 

Qj total probability of failure of / h redundant part 
n total number of redundant parts 





Figure 3-7.— Space capsule guidance model. 


Example 9: Find the reliability of the redundant system 
shown in Figure 3-8. 

Solution 9: Step 1— Solve for the reliability of parts 1 and 2. 
/?, = e -l(i 2 o/io 6 )xio 3 i _ e -o.i 20 _ o.887 

R. = e -h'2 = f -|(340/l()0|XlO-''| _ g -0.340 _ Q.7J2 


^simple redundant — 1 Q\Q 2 ~ 1 (0. 1 13)(0.288) 

= 1 - 0.033 = 0.967 

There is a 96.7 percent chance, therefore, that both parts will 
not fail during the 1000-hour operating time. 

Compound redundancy.— Compound redundancy exists 
when more than one of n redundant parts must succeed for 
the system to succeed. This can be shown in a model of a three- 
element redundant system in which at least two of the elements 
must succeed, as shown in figure 3-9. 

From probability theorem 3, chapter 2, the possible events 
are 



Figure 3-8.— Simple redundancy model using failure rates and operating times. 


Step 2— Solve for the unreliability of each part. 

Q\ = 1 -R\ =0.113 
Q 2 = 1 - R 2 = 0.288 

Solve for the reliability of the redundant system. 
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Figure 3-9.— Compound redundancy model. 


^ I ^ 2^3 + ^ 1^203 + ^ 102^3 + 01 ^ 2^3 + ^10203 

+ Qi02^3 + + C 1 C 2 G 3 

To simplify the notation, let /?, = /? 2 = /? 3 and 0, = Q 2 = 0 3 . 
This reduces the expression to 

R 3 + R 2 Q + R 2 Q + R 2 Q + RQ 2 + RQ 2 + RQ 2 + Q : 

or 

/? 3 + 3/? 2 0 + 3 RQ 2 + Q 3 

Because the sum of these probabilities equals unity and at least 
two of the three parts must succeed, the probability for success 
is given by 

R s = R 3 + 3 R 2 Q = 1 - QRQ 2 + 0 3 ) 

where 3 RQ 2 represents one part succeeding and two parts 
failing and Q 3 represents all three parts failing. 

Example 10: Assume that there are four identical power 
supplies in a fire control center and that at least two of them 
must continue operating for the system to be successful. Let 
each supply have the same reliability, R - 0.9 (which <”>uld 
represent e -x ' or R, or R). Find the probability of system 

SUCCeSS ^si m p| c redundant - 


Solution JO: The number of possible events is given by 

(R + Q) 4 = R A + 4R 3 Q + 6 R 2 Q 2 f 4 RQ 3 + Q 4 

The sum of the probabilities of these events equals unity; 
therefore, the expression for two out of four succeeding is 

R S = R 4 + 4R 3 Q + 6 R 2 Q 2 = 1 - (4 RQ 3 + Q 4 ) 
Substituting R — 0.9 and Q— 1 — 0.9 gives 
R s = 1 - (4*G 3 + Q 4 ) = 1 - [4(0.9)(0. 1) 3 + (0.1) 4 ] 

= I - [(3.6X0.001) + 0.0001] = 1 - (0.0036 + 0.0001) 
= 1 - 0.0037 = 0.996 

Calculation of Reliability for Complete System 

To find the reliability for a complete system, begin by 
developing a model for the system, write the equation for the 
probability of success from the model, and then use the failure 
rates and operating times of the system elements to calculate 
the reliability of the system (refs. 3-6 to 3-8). 

Example 11: Consider the system model with series and 
redundant elements shown in figure 3-10. 

Solution 11: The equation can be written directly as 

Rs = R&RjO - QsQsQd 

where R\RiRt, represents the probability of success of the series 
parts and (1 - QiQ^Qt,) represents the probability of success 
of the three parts in simple redundancy. If we know that 


/?, = 0.99 = e * 001 

R 4 = 0.85 

* 2 = 0.999 = <T° 001 

/? 5 = 0.89 

/?3 = 0.95 = e' 005 

R 6 = 0.78 


where R may represent <? -x \ inherent reliability if,-, or 
observed product reliability depending on the stage of product 



Figure 3-10.— Model of system with series and redundant elements. 
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development, then the reliability of the system is 
R s = e~ 00 ' e -0001 e _005 [ I - (I - 0.85)(1 - 0.89)(1 - 0.78)] 
= e' 006l [l - (0.l5)(0.11)(0.22)] = <>- 006l (l - 0.00363) 

_ e -0.061 e -0.0036 _ e -0.065 _ q 935 

However, this does not mean that there will be no equipment 
failures. The system will still succeed even though one or two 
of the redundant paths have failed. 

Example 12: Write the equation for the system shown in 
figure 3-11. 

Solution 12: The equation can be written directly as 
R, •■= *,*,[ 1 - (RiQ 4 Q 5 + 0 3 /f 4 e 5 + 030 4 * 5 

+ ftGa&Md ~ 0 6 G 7 ) 


where *|* 2 is the probability that the two parts in series will 
not fail, 1 - (*30405 + . . . + 03040s) is the probability 
that two out of three of the compound redundant parts will 
not fail, and (I - 0 6 0 7 ) is the probability that both of the 
simple redundant parts will not fail. If data giving the 
reliabilities of each part are available, insert this information 
into the system success equation to find the system reliability. 

Example 13: Write the equation for the system shown in 
figure 3-12. 

Solution 13: The equation can be wriuen directly as 

R s = *,*6*7(1 - [0 2 03d - *4* 5 )]] 

where *1*6*7 is the reliability of the series paits, (1 - *4*5) 
is the probability that * 4 or * 5 will fail in the bottom 
redundant path, and [1 - (0203(1 — *4*5)]) is the reliability 
of the three paths in simple redundancy. 



Part 6 does 


Parti does land I Part 2 does land if I fail 


I Success 


Part 7 does 
not fail 


Figure 3-11.— System reliability model using series, simple redundancy, and compound redundancy elements. 


Part 2 does 
not fail 


Parti does I and If I Part 3 dees 

not fall I I not fail 


and if I Parts6and7 I then I _ 
do not fail ' 





Part 4 does 
not fail 


Part 5 does 
not fail 


Figure 3-12.— Model with series elements in redundant paths. 






Concluding Remarks 

Chapter 3 has presented several inportant concepts that you 
should have clearly in mind: 

(1) The exponential distribution e~ K ' represents the prob- 
ability that no catastrophic part failures will occur in a product. 

(2) The failure rate X as used in e ^ is a constant and 
represents the rate at which random catastrophic failures occur. 

(3) Although the cause of failure is known, random failures 
may still occur. 

(4) The mean time between failures (MTBF) is the recip- 
rocal of the fail -e rate. 

(5) In reliability, devices are in series if each one is required 
to operate successfully for the system to be successful. Devices 
are parallel or redundant if one or more can fail without 
causing system failure but at least one of the devices must 
succeed fer the system to succeed. 

In addition, you should be able to calculate the following: 

(1) The reliability of a device, given failure rate and 
operating time. 

(2) The reliability of dev ices connected in series from the 
product rule: 

n 

/?, = n Rj 

j = i 

(3) The reliability of devices connected in simple redun- 
dancy from 


^simple redundant 1 U Qj 

j= I 

(4) The reliability of n devices connected in compound 
redundancy by expanding (R + Q) n and collecting the 
appropriate terms. 


And finally, you should be able to combine the four methods 
described above to calculate the reliability of a total system. 

In 1985, alternative methodologies were introduced in the 
form of computer reliability analysis programs. One such 
underlying model uses a Weibull failure rate during the burn- 
in, or "infant mortality,” period and a constant failure rate 
during the steady-state period for electronic devices, initial 
results indicate that given a 15- to 40-year system life the infant 
mortality period is assumed to last for the first year. Of course, 
the higher the stress of the environment, the shorter the infant 
mortality period. The point is that there are many ways of 
performing reliability studies, and different methodologies 
could be equally appropriate or inappropriate. Appendix C 
describes five distribution functions that can be used for 
reliability analysis. Table C-l shows the time to failure fit 
for various systems. The basic criteria relate to the distribution 
of failures with time. 
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Reliability Training' 

la. Of 45 launch vehicle flights, 9 were determined to be failures. What is the observed reliability? 
A. 0.7 B. 0.8 C. 0.9 

lb. What is the observed reliability if the next five flights are successful? 

A. 0.72 B. 0.82 C. 0.87 

l c. After the five successes of part lb, how many more successes (without additional failures) are required 
for a reliability of R = 0.90? 

A. 20 B. 30 C. 40 

2. A three-stage launch vehicle has a reliability for each stage of /?, = 0.95, R 2 = 0.94, /?, = 0.93. 

a. What is the probability of one successful flight? 

A. 0.83 B. 0.85 C. 0.87 

b. What is the probability of flight failure for part a? 

A. 0.00021 B. 0.15 C. 0.17 

c. What is the probability of two successful flights? 

A. 0.689 B. 0.723 C. 0.757 

3. You are taking a trip in your car and have four good tires and a good spare. By expanding (R + Q) \ 

a. How many events (good tires or flats) are available? 

A. 16 B 32 C. 64 

b. How many combinations provide four or more good tires? 

A. 6 B. 7 C. 16 

c. If R = 0.99 for each tire, and a successful trip means you may have only one flat, what is the 
probability that you will have a successful trip? 

A. 0.980 B. 0.995 C. 0.9990 

4. A launch vehicle system is divided into five major subsystems, three of which have already been built 
and tested. The reliability of each is as follows: /?, = 0.95. R 2 = 0.95, /?, = 0.98. The reliability of 
the overall system must be equal to. or greater than, 0.85. What will be the minimum acceptable reliability 
of subsystems 4 and 5 to ensure 85-percent reliability? 

A. 0.92 B. 0.95 C. 0.98 

5a. A launch vehicle test program consists of 20 test firings requiring 90-percent reliability. Five tests 
have already been completed with one failure. How many additional successes must be recorded to 
successfully complete the test program? 

A. 13 B. 14 C. 15 

5b. Based on the probability (four successes in five flights) what is the probability of achieving successful 
completion of the test program? 

A. 0.04 B. 0.167 C. 0.576 


'Answer* arc given al the end of this manual. 


6. During individual tests of major launch vehicle subsystems, the reliability of each subsystem was found 
to be 

Subsystem I =0.95 
Subsystem 2 = 0.99 
Subsystem 3 = 0.89 

Subsystem 4 = 0.75 j 

Since all subsystems are required to function properly to achieve success, what increase in reliability 
of subsystem 4 would be necessary to bring the overall system reliability to 0.80? 

A. 15 percent B. 20 percent C. 25 percent i 

7. Solve for the following unknown values: 

! 

a. X = 750 x 10 * failures/hour; t = 10 hours; R = ? j 

A. 0.9925 B. 0.9250 C. 0.9992 I 

I 

i 

b. X = 8.5 percent failures/ 10 3 hours; / = 3000 hours; R = ? j 

A. 0.9748 B. 0.7986 C. 0.0781 


c. MTBF = 250 failures/hour; / = 0.5 hour; R = ? 

A. 0.9802 B. 0.9980 C. 0.9998 

d. R = 0.999; / = 10 hours; X = ? 

A. 1000 x 10 failures/hour B. 10 x 10 -6 failures/hour C. 10 percent failures/10 3 hours 

e. MTBF = ? 

A. 10 4 failures/hour B. 10 s failures/hour C. 10 6 failures/hour 

8. The a priori MTBF ficd«-tion of a printed circuit board was 12.5 x I0 6 hours. Find the number of 
expected failures during a 10 8 -hour (accelerated) life test of ID circuit board samples. 

A. 12.5 B. 80 C. 125 

9a. Write the reliability equation for the battery activation success diagram shown below: 



If 
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And 
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Ignites 
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activate 
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(part 7) 
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9b. If R • 0.9 for all series and R = 0.8 for all parallel parts, solve for R,. 

A. 0.73 B. 0.26 C. 0.67 

10. A launch vehicle subsystem is required to be stored for 10 years (use 9000 hours = 1 year) If the 
subsystem reliability goal is 0.975. 

a. What X is required with no periodic checkout and repair? 

A. 2800 x 10~ 9 B. 28 x 10' 9 C. 280 X 10" 9 

b. What X is required with checkout and repair every 5 years? (Assume 100-percent checkout.) 

A. 5600 x 10" 9 B. 56 x 10~ 9 C. 560 X 10~ 9 

c. What X is required with checkout and repair every year? (Assume 100-percent checkout.) 

A. 2800 x 10~ 9 B. 28 x 10 -9 


C. 280 x I0 -9 






Chapter 4 

Using Failure Rate Data 


Now that you have a working knowledge of the exponential 
distribution e~ * and have the fundamentals of series and 
redundant models Firmly in mind, the next task is to relate 
these concepts to your everyday world. To do this, we explore 
further the meaning of failure rates, examine variables that 
affect part failure modes and mechanisms, and then use part 
failure rate data to predict equipment reliability. We introduce 
a simple technique for allocating failure rates to elements of 
a system. The concepts discussed in this chapter are tools the 
designer can use for trading off reliability with other factors 
such as weight, complexity, and cost. These concepts also 
provide guidelines for designing reliability into equipment 
during the concept stage of a program. 

Variables Affecting Failure Rates 

In chapter 3 failure rate X was defined as a constant in time 
representing the rate of occurrence of random catastrophic 
failures in the equipment. An actual observation of a constant 
failure rate is shown in Figure 4-1. The results of two tests 
are shown in this Figure. One is an operating life test lasting 
4500 hours; the other, a storage test lasting 7000 hours. Each 
test is discussed separately. 

Operating Life Test 

The tests involved 7575 parts— 3930 resistors, 1545 
capacitors. 915 diodes, 1080 transistors, and 105 transformers. 
One-third of the parts were operated at -25 ’F, one-third at 
77 *F, and one-third at 125 # F. The parts, tested in circuits 
(printed circuit boards), were derated no more than 40 percent. 
The ordinate of the curve shows cumulative failures as a 
function of operating time. For example, at about 240 hours 
the First failure was observed, at about 385 hours the second, 
etc. Several important observations can be made concerning 
failure rates and failure modes. 

Constant failure rate .— Figure 4-1 shows that the failure 
rate for the First 1600 hours is constant at one failure every 
145 hours. This agrees with the constant-X theory. Bear in 
mind that constant failure rate is an observation and not a 
physical law. Depending on the equipment, failure rates may 
decrease or increase for a period of time. 


Random nature .— Notice that the failures in this constant- 
failure-rate region are random (in occurrence). For example, 
two diodes fail, then three transistors, then a silicon switch, 
then a diode, then a trimpot and a resistor, etc. 

Repetitive failures .— Figure 4-1 also shows that during the 
first 1600 hours only two of these failures involved the same 
type of device. This is important because in most systems the 
problems that get the most attention are the repetitive ones. 
It should be apparent in this case that the repetitive failures 
are not the ones that contribute the most to unreliability (failure 
rate). And taking corrective action on the repetitive type 
of failure would only improve the observed failure rate by 
18 percent. 

Failure modes. —Table 4-1 shows the observed failure 
modes (the way the failures were revealed) for the transistor, 
diode, and resistor failures given in Figure 4-1. Note in 
table 4- 1(a) that the short failure mode for transistors had an 
occurrence rate five times that of any other mode. Note also 
that the eight transistor failures were distributed about evenly 
in the three environments but that some different failure modes 
were observed in each environment. 

Observe again in table 4- 1(b) that the short failure mode 
for diodes occurred most frequently. The failures were not 
distributed evenly in each environment, but a different failure 
mode occurred in each environment. 

Resistors failed in two modes (table 4- 1(c)): one intermittent 
resistor at low temperatures and one tolerance failure at high 
temperatures. 

Bum-in .— As shown in figure 4-1 after 1600 hours the 
failure rate of the 7575 parts dropped by a factor of 7 for the 
remaining 2900 test hours (3 failures per 2900 hours, failures 
12, 13, and 14, as compared with 1 1 failures per 1600 hours). 
This is an example of what are commonly called bum-in 
failures. The first 1 1 failures represent parts that had some 
defect not detected by the normal part screening or acceptance 
tests. Such defects do not reveal themselves until the part has 
been subjected to operation for some time. As mentioned 
earlier, eliminating the repetitive failure would only decrease 
the failure rate in the first 1600 hours by about 18 percent, 
but if screening tests were sensitive enough to detect all defects, 
the failure rate would approach the intrinsic failure rate shown 
in Figure 4- 1 right from the start. 
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Figure 4-1.— Observed part failures versus lest and storage time. 


In summary, some of the observed properties of operating 
failure rates are as follows: 

(1) For complex equipment the intrinsic failure rate of 
electronic parts is usually constant in time. 

(2) Failures are random, with repetitive failures repre- 
senting only a small portion of the problems. 

(3) Failure modes of parts and equipment vary , depending 
on the operating environment. 

(4) Most parts have a dominant failure mode. For example, 
the dominant failure mode for semiconductors is shorting. 

(5) Rigid part screening and acceptance criteria can sub- 
stantially reduce operating failure rates by eliminating early 
failures. 

Storage Test 

After the operating test the parts were put in storage for 
approximately 7000 hours (10 months) «nd then retested 
to determine the effect of storage on parts. As shown in 
figure 4-1, three failures (14, 15, and 16) were observed at 
the end of the storage period. Note that the average failure 
rate observed in storage (one failure per 2300 houts) is close 
to the same rate observed in the previous 2900 hours of 
operation. Thus, it can be concluded that storage does produce 
part failures and that the storage failure rate may be as high 
as the operating rate. Industry is conducting a great deal of 


research on this problem because storage failure rates become 
a significant factor in the reliability of unmanned systems and 
affect considerably the maintenance policy of manned systems. 

Summary of Variables Affecting Failure Rates 

Part failure rates are thus affected by 

(1) Acceptance criteria 

(2) All environments 

(3) Application 

(4) Age or storage 

To find ways of reducing the occurrence of part failures, we 
observe failure modes, learn what caused the failure (the failure 
stress), determine why it failed (the failure mechanism), and 
then take action to eliminate the failure. For example, one of 
the failure modes observed during the storage test was an 
“open” in a wet tantalum capacitor. The failure mechanism 
was deterioration of the end seals, which allowed the 
electrolyte to leak. One obvious way to avoid this failure mode 
in a system that must be stored for long periods without 
maintenance is not to use wet tantalum capacitors. If this is 
impossible, the next best thing would be to redesign the end 
seals. This would no doubt require further testing to isolate 
the exact failure stress that produces the failure mechanism. 
Once isolated, the failure mechanism can often be eliminated 
through redesign or additional process controls, 


36 


sitmmtiii - 











TABLE 4-1.— FAILURE MODES 
(a) Transistors 


TABLE 4-3. -STRESS RATIOS THAT MEET 
ALLOCATION REQUIREMENT 


Observed 

part 

failure 

mode 

Temperature 

. *F 

Total 

Observed 

-25 

77 

125 

failures 

failure 

rate. 

failurrs/hr 

Open 

Shon 

MD-90 

2N498 

2N389 

2N396 

MD-90 

2N1016B 

1 

5 

0.206/ 10 6 
1.03/10** 

Intermittent 

Leakage 

MD-90 


.206/10** 

.206/10^ 

2 N 1057 


1 

I 

Totals 

3 

2 

3 

8 

1.65/I0 6 


(b) Diodes 


Open 

Short 

— 

1N76I 

IN708A 

SA60A 

1N483 

1 

3 

0.24/10* 

.73/10* 

Totals 

0 

3 

« 

4 

0.97/10* 


■ • i 

(c) Resistors 

Intermittent 

Tolerance 

Trimpot 


— 
Metal film 

1 

l 

^006/10** 
.06/ 10 6 

Totals 

1 

0 

1 

2 

0.12/10* 


Part 

temperature. 

•c 



Stress ratio. W 



0.1 

0.2 

0.3 

0.4 

0.5 

0.6 


— 

Failure rate of derated part per I0 6 hr. 

*/> 

30 

40 

50 

60 

70 

0.25 

0.25 

0.24 

0.24 

0.23 

0.22 


* 


table 4-2.— failure rate calculation 

(a) Tactical fire control station logic gate 


Component 

Stress ratio 

Number 

uscu. 

N 

Failure rate 
of derated 
part at 
40 *C 
A 0 , 

failures/ 10** hr 

Application 
factor for 
vehicle, 
ground 
mounted. 

Ka 

Total failure 
rate. 

V = vx,a,. 

failures/IO* hr 

Resistor, composition (2000 fl) 
Resistor, composition (180 000 fl) 
Resistor, composition (22 000 fl) 
Resistor, composition (6500 fl) 
Transistor, germanium (PNP type) 

Diode. italA 

0.5 

.5 

.6 

.5 

<1 W; 0.4 normalized 
junction temperature 
.3 

1 

1 

1 

2 

1 

1 

0.0035 

.0035 

.0038 

.0035 

1.3 

3.5 

10 

J 

8 

5 

0,035 

.035 

,038 

.070 

10.400 

17.500 


Toui, \ a 

> 2 \ r = 29.68 


(b) Proposed logic gate 

Resistor, film (1300 fl) 
Resistor, film (3320 fl) 
Resistor, film (46 600 fl) 
Transistor, silicon (NPN type) 

Diode. IN3IA 

0.8 

.2 

.2 

<1 W; 0. 15 normalized 
junction temperature 
.2 

1 

3 

0.19 

.14 

.14 

.165 

3,0 

0,3 

,3 

.3 

8 

5 

0.057 

.042 

.042 

1.320 

75.000 


Total. \ - 

C \ r « 76.46. 
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One of the best known methods of representing part failures 
is the use of failure rate data. Figure 4-2 (from ref. 4- 1 ) shows 
a typical time-versus-failure-rate curve for flight hardware. 
This is the well-known ‘‘bathtub curve/’ which over the years 
has become widely accepted by the reliability community. It 
has proven to be particularly appropriate for electronic equip- 
ment and systems. It displays the sum of three failure rate 
quantities: quality (QFR), stress (SFR), and wearout (WFR). 

Zone 1 , the infant mortality period, is characterized by an 
initially high failure rate (QFR). This is normally the result 
of poor design, use of substandard components, or lack of 
adequate controls in the manufacturing process. When these 
mistakes are not caught by quality control operations, an early 
failure is likely to result. Early failures can be eliminated by 
a “burn-in'‘ period during which time the equipment is 
operated at stress levels closely approximating the intended 
actual operating conditions. The equipment is then released 
for actual use only when it has successfully passed through 
the bum-in period. For most well-described complex 
equipment, a 100-hour failure-free bum-in is usually adequate 
to cull out a large proportion ot the infant mortality failures 
caused by stresses on the parts. 

Zone II, the useful life period, is characterized by an 
essentially constant failure rate (SFR). This is the period 
dominated by chance failures. Chance failures are those 
failures that result from strictly random or chance causes. They 
cannot be eliminated by either lengthy bum-in periods or good 
preventive maintenance practices. 

Equipment is designed to operate under certain conditions 
and to have certain strength levels. When these strength levels 
are exceeded because of random unforeseen or unknown 
events, a chance failure will occur. Although reliability theory 
and practice are concerned with all three types of failure, the 
primary concern is with chance failures, since they occur 
during the useful life of the equipment. Figure 4-2 is somewhat 
deceiving because zone II «s usually much longer than zone 
I or III. The time when a chance failure will occur cannot be 
predicted, but the likelihood or probability that one will occur 
during a given period of time within the useful life can be 
determined by analyzing the equipment design. If the proba- 


Equipment life periods 



Figure 4-2.— Hazard rate versus equipment lift periods. 


bility of a chance failure is too great, either design changes must 
be introduced or the operating environment made less severe. 

The SFR period is the basis for the application of most 
reliability engineering design methods. Because it is coastant, 
the exponential distribution of time to failure is applicable and 
is the basis for the design and prediction procedures spelled 
out in documents such as MIL-HDBK-2I7E (ref. 4-2). 

The simplicity of the approach (utilizing the exponential 
distribution, as previously indicated) makes it extremely 
attractive. Fortunately, it is widely applicable for complex 
equipment and systems. If complex equipment consists of 
many components, each having a different mean life and 
variance that are randomly distributed, then the system 
malfunction rate becomes essentially constant as failed parts 
are replaced. Thus, even though the failures might be wearout 
failures, the mixed population causes them to occur at random 
intervals with a constant failure rate and exponential behavior. 
This has been verified for much equipment from electronic 
systems to rocket motors. 

Zone ID, the wearout period is characterized by an increasing 
failure rate (WFR) as a result of equipment deterioration due 
to age or use. For example, mechanical components, such as 
transmission bearings, will eventually wear out and fail 
regardless of how well they are made. Early failures can be 
postponed and the useful life extended by ^ood design and 
maintenance practices. The only way to prevent failure due 
to wearout is to replace or repair the deteriorating component 
before it fails. 

Because modem electronic equipment is almost completely 
composed of semiconductor devices that really have no short- 
term wearout mechanism, except for perhaps electromigration, 
one might question whether predominantly electronic equip- 
ment will even reach zone III of the bathtub curve. 

Different statistical distributions might be used to charac- 
terize each zone. Hazard rate has been defined for five different 
failure distribution functions, see figure C- 1 in the appendix. 
Depending on which distribution fits the hazard rate data best, 
a failure distribution function cun be selected. The infant 
mortality period for the typical hazard rate in figure 4-2 might 
be represented by the Weibull distribution, the useful life 
period by the exponential distribution, and the wearout period 
by the log normal distribution. 


Part Failure Rate Data 

It is common in the field of reliability to represent part 
integrity or reliability in terms of failure rate or mean time 
between failures (MTBF). In general, part failure rates are 
presented as a function of temperature and electrical stress as 
shown in figure 4-3. The family of curves on the graph 
represents different applied electrical stresses in terms of a 
stress ratio or derating factor. For example, if a part is to 
operate at temperature A and is derated 20 percent (stress ratio, 
0.8), that part will have a failure rate of X = 0.8 as shown. 
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Figure 4-3.-Failure rate versus electrical stress ratio and temperature. 


If the part is derated 70 percent (stress ratio, 0.3), the part 
will have a failure rate of X = 0.3, etc. Failure rate is usually 
given in failures per 10 6 hours, although as indicated in 
chapter 3 other dimensions are used depending on who 
publishes the data. 

The current authoritative failure rate data published by the 
Department of Defense are in MIL-HDBK-2 17E (ref. 4-2). 
The MIL-HDBK-217 series is a direct result of the 1952 
AGREE effort mentioned in chapter i . The publications listed 
in table 1 - 1 as well as references 4-3 to 4-5 are also offshoots 
of this effort to meet the need for authoritative, statistically 
based part failure rates. Because new data on both existing 
and new state-of-the-art parts are constantly being generated 
and analyzed, failure rate handbooks do change. Therefore, 
be sure to use the latest version available. Even the latest 
version of the data used for compiling the handbook may not 
represent the parts you are using. The best procedure is to 
use your own failure rate data. 

As emphasized in chapter 3 failure rates are statistical, and 
there is no such thing as an absolute failure rate. Consider 
the simple definition of failure rate: 

^ _ Number of observed failures 
Total operating time 

Obviously, if today we observe two failures in 100 hours and 
tomorrow we accumulate no more failures, the new failure 
rate is two failures in 124 hours. Then, if a failure occurs in 
the next 1-hour period, the failure rate is three failures in 125 
hours. Therefore, we can never know what the true failure 
rate is, but we can determine representative failure rates or 
best estimates from many hours of observed operating 


time. This type of failure rate data is presented in the 
MIL-HDBK-217 series. 

Improving System Reliability Through 
Part Derating 

The best way to explain how to derate a component is to 
give an example. Consider two 20-V wet slug tantalum 
capacitors, both to be operated at a component temperature 
of 60 "C. One is to be operated at 20 V and the other at 12 V. 
First, find the stress ratio or operating-to-rated ratio for 
both applications: 

Stress ratio =^ ratingvoltage 
Rated voltage 

Hence, one capacitor has a stress ratio of 1 .0, 


Stress ratio = =1.0 

20 V 

and the other, a stress ratio of 0.6, 

12 V 

Stress ratio = = 06 

20 V 

(A stress ratio of 0.6 means the same as "derating” the 
component 40 percent.) To find the failure rate X for each 
capacitor, go to the MIL-HDBK-2 17E (ref. 4-2) table for 
MIL-C-3965 glass-sealed wet slug capacitors. Move 
horizontally across the 60 °C line to the vertical 0.6 and 1.0 
stress ratio columns and read directly: 

Xo.6 = 0.12 failure per 10 6 hours 
X| o = 0.57 failure per 10 6 hours 

As mentioned earlier, failure rates are not absolute: 
therefore, the failure rates just calculated for the two capacitors 
are not absolute. In other words, we cannot state definitely 
that one will fail at the rate of 0.12 per 10 6 hours and the 
other at 0.57 per 10 6 hours when used in the system. We can 
say, at least, that the nonderated capacitor is expected to have 
a failure rate 4.75 times that of the derated one. If we derated 
still further, say 90 percent, Xo., = 0.0013/10 6 , we could 
expect the capacitor to be 438 times more reliable than the 
nonderated capacitor. This is, of course, the reason for derating 
in the first place. 

The same failure rate information is presented on the 
opposite page of MIL-HDBK-2 17E in figure 4-3 format. 
Although the X values must be approximated from the curves, 
this form of presentation shows graphically the effects of 
temperature and stress on failure rate and also the effect of 
not derating. 


Use of Application Factor 

Thus far only the stress ratio and the ambient temperature 
of the component have been considered in the derated failure 
rate \ D . However, other stresses, such as vibration, shock, 
and humidity, also affect failure rate. These environmental 
factors are taken into account by assigning a weighting 
application factor K A . Thus, the total failure rate \ r becomes 

The K a varies from component to component and by 
application. MIL-HDBK-217E (ref. 4-2) lists five 
applications: ground, vehicle mounted ground, shipboard, 
airborne, and missile. Thus, if our two capacitors are used 
in a missile system, their failure rates become 

Xo.f, = 0. 12/10 6 x 25 = 3.0/10 6 
X, o = 0.57/10 6 x 25 = 14.25/ 10 6 


proposed components are higher and the failure contribution 
of the five diodes alone is double the total failure rate of the 
tactical circuit. 

These calculations are for an operating circuit. Now consider 
the effects of a nonoperating circuit on the mission model. 
From figure A-3 in appendix A, the operating application 
factor for ground electronics equipment is given as 5x I0\ 
The nonoperating application factor is 8xl0 2 . The scale 
factor for a nonoperating circuit using operating failure rates 
is given by 





0.8x 10 

5.0XJ0 3 


The expected failure rate for a nonoperating circuit is given by 
A, = K s £ A, = 0. 16 x 29.68/ 10 6 

= 4.75 failures/IO 6 hours 


The K A factor includes the failure rate for the connection 
technique normally associated with that part class, except for 
wires and cables. 

Predicting Reliability From Part Failure Rate Data 

We have shown so far that the failure rate of a part is given 
by A dK a and, as shown in chapter 3, the reliability of a part 
used in a circuit or system can be estimated from R = e~*. 
Further, we can estimate the reliability of a system from 


The operating and nonoperating times during a mission are 
used in the model to calculate reliability. 

The reliability of either circuit, operating or not, as discussed 
in chapter 3, would be given by 

R, = e'^ 

where 

A,, circuit failure rate 

operating time of circuit 


where 

A, failure rate of /' lh part 
/, operating time of i ,h part 

This is also discussed in chapter 3. 

For example, table 4-2(a) shows a reliability estimate for 
a tactical fire control station logic gate. The total failure rate 
of each part type in the circuit is shown as \ T - NX pK A . The 
expected failure rate of the circuit A f is then found from 

n 

A r = E A, = 29.68 failures/10 6 hours 

i = l 


Predicting Reliability by Rapid Techniques 

The preceding logic gate illustration is an example of a relia- 
bility prediction based on detailed knowledge of parts population 
and stress. In many situations, however, this type of detailed 
prediction is not possible. Some situations that come to mind 
arc concept and tradeoff studies where detailed parts counts 
are not available, where operating stress levels have not been 
determined, and where time or personpower is limited. 
Fortunately, a number of rapid reliability prediction techniques 
are available. One good technique is presented in detail in 
MIL-HDBK-2 17E (ref. 4-2). Usually, one or more of these 
techniques can be used. Although the results lack the detail 
of the logic gate example, these methods aid in quickly screening 
candidate designs and help managers make sound decisions. 


The reliability estimate for a logic gate proposed for another 
system is shown in table 4-2(b). Note that the complexity 
(number of parts) is higher for the proposed circuit than for 
the tactical circuit by a ratio of 9:7 and the estimated failure 
rate is higher by a factor of 2.6. This is possible because, in 
spite of greater derating, the failure rates of most of the 


Use of Failure Rates in Tradeoffs 

The failure rate tables and derating curves are useful from 
the designer’s point of view because they provide knowledge 
for making reliability tradeoffs and permit a more practical 
method of establishing derating requirements. For example, 
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suppose we have two design concepts for performing some 
function. If concept A is found to have a failure rate that is 
10 times higher than that of concept B, it can be expected that 
concept B will fail one-tenth as often as concept A. If it is 
desirable to use concept A for other reasons, such as cost, 
size, performance, or weight, the derating failure rate curves 
can be used to improve concept A’s failure rate (e.g., select 
components with a lower failure rate, derate the components 
more, or both). An even better approach is to find ways to 
reduce the complexity and thus the failure rate of concept A. 

As another example of the use of failure rate data in 
tradeoffs, consider figure 4-4. This figure gives a failure-rate- 
versus-temperature curve for the electronics of a complex (over 
35 000 parts) piece of ground support equipment. The curve 
was developed as follows: 

(1) A failure rate prediction was performed by using 
component failure rates and their application factors K A for 
an operating temperature of 25 *C. The resulting failure rate 
was chosen as a reference point, as indicated on the curve. 

(2) Predictions were then made by using the same method 
for temperatures of 50, 75, and 100 6 C. The ratios of these 
predictions to the reference point, 25 *C, were plotted versus 
component operating temperature, with the resulting curve for 
the equipment. This curve was then used to provide tradeoff 
criteria for using air-conditioning versus blowers to cool the 
equipment. To illustrate, suppose the maximum operating 
temperatures expected are 50 *C with air-conditioning and 
75 *C with blowers. Suppose further that the required failure 
rate for the equipment, if the equipment is to meet its reliability 
goal, is one failure per 50 hours. A failure rate prediction at 
25 °C might indicate a failure rate of one per 100 hours. 
Re r erring to figure 4-4, we see that the maximum allowable 


operating temperature is therefore 60 “C, since the maximum 
allowable failure rate ratio is X = 2. In other words, at 60 *C 
the equipment failure rate will be (1/100) x 2 = 1/50, which 
is the required failure rate. If blowers are used for cooling, 
the equipment must operate at temperatures as high as 75 # C; 
if air-conditioning is used, the temperature need not exceed 
50 °C. Therefore, it would appear that we must use air- 
conditioning if we are to meet the reliability requirement. 

But other factors must be examined before we arrive at a 
final decision. Whatever type of cooling equipment is selected, 
total rystem reliability now becomes 

R t = /?,/?, 

Therefore, the effect on the system of the cooling equipment's 
reliability must be calculated. An even more important con- 
sideration is the effect on system reliability should the cooling 
equipment fail. Because temperature control appears to be 
critical, loss of temperature control may have serious system 
consequences. Therefore, it is too soon to rule out blowers 
entirely. A failure mode, effects, and criticality analysis 
(FMECA) must be made on both cooling methods to examine 
all possible failure modes and their effects on the system. Only 
then will we have sufficient information with which to reach 
a sound decision. 


Nonoperating Failures 

As pointed out in discussing figure 4-1, parts continue to 
fail when not in use. These nonoperating failures are converted 
to nonoperating failure rates. In general, electronic parts fail 
less frequently in the nonoperating mode than in the operating 
mode. Certain hydraulic and mechanical parts, however, fail 
more frequently in the nonoperating mode. For many military 
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Figure 4-4.-Predicted failure rale ratios versus temperature for ground 
support equipment (electronics). 


weapon systems the nonoperating role is the norm. Missiles 
may remain in storage depots or in a dormant standby condition 
for months or years before being fired. Likewise, many 
subsystems in orbiting satellites are passive most of the time. 
In these cases, system reliability becomes 
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Because nonoperating time t m can be many orders of magnitude 
greater than operating time t,„ nonoperating failures often 
represent a major portion of total system failures. There is, 
hence, increased interest in how, why, and at what rate non- 
operating parts fail. Some recent studies gave indications that 
nonoperating failure rates may not be as high as some hand- 
books might indicate. Turn-on and test stress failures affect 
the count of true nonoperating failures. 
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Applications of Reliability Predictions to 
Control of Equipment Reliability 

Even though we have indicated that reliability predictions 
do not give absolute answers, several things can be done to 
make these predictions more meaningful. Consider the concept 
stage— the most important stage to reliability because the 
potential reliability of the system is fairly well defined by the 
time the concept is selected. To predict the potential reliability, 
we usually must 

(1) Predict the number and types of parts to be used in the 
system 

(2) Choose a part derating factor 

(3) Choose a maximum operating environment 

Now, to make the prediction meaningful, we must 

(1) Place a complexity limit (the limit predicted in (1) above) 
on the system 

(2) Direct the minimum amount of derating allowed 

(3) Approve part applications to ensure that parts are used 
in the correct manner and will be operating within the 
assumed environmental limits 

Standardization as a Means of Reducing 
Failure Rates 

Another means of establishing control over the failure 
rate (reliability) of a product is to employ standardization 
principles. As an extreme illustration, suppose we need 1000 
transistors for a system and allow each transistor to be a 
different type, bought from a different vendor. If each vendor 
part has five failure mechanisms peculiar to that vendor, the 
system will have 5 x 1000 = 5000 failure mechanisms. If, 
through testing, we find one failure mechanism and eliminate 
it, we have reduced the failure mechanisms of the system by 
a factor of only 1 /5000. If, on the other hand, we could require 
that the 1000 transistors be of th'. same type and bought from 
the same vendor and if this vendor's part has five failure 
mechanisms, the system also has only five ways to fail. If we 
then eliminate one of these failure mechanisms by testing, we 
have reduced the failure mechanisms of the system by one- 
fifth, or 20 percent. You can readily see, though, that an initial 
reliability prediction would be the same in both cases because 
each system uses 1000 transistors. Also the chance of observing 
the failure mode will increase by five times. Quick failure 
mode detection and correction is important in reliability work. 


This apportioning process is called allocation in reliability 
engineering. 

Ir a similar fashion the reliability organization usually 
allocates the system reliability or failure rate requirements only 
to the assembly or subassembly level. The designers, therefore, 
must allocate goals to the part level for the component for 
which they are responsible. All allocations at any level are 
performed in such a manner that, when the failure rates or 
reliabilities of the system elements are combined (by using 
the prediction methods discussed in chapter 3), the goal or 
requirement for the system is obtained. The allocation process, 
together with part failure rate data, provides the designer with 
a method for determining how good the parts must be if the 
design is to meet the specified reliability. 

The first method of allocating failure rates is called the 
assembly method. If the reliability requirement of a system, 
subsystem, or assembly, as well as the operating time interval, 
is known, the required failure rate may be calculated from 

The resulting failure rate can then be divided by the antic- 
ipated number of parts to be used to allocate the average failure 
rate requirement down to the part level. 

Example 1: Consider a missile that has a reliability require- 
ment of 0.99 for a flight period of 0.5 hour. The estimated 
complexity of the missile is 10 000 active parts. Find the 
average failure allocation for each part. 

Step 1— Write the reliability equation for the missile. 



ABocatlor. of Failure Rates and Reliability The , ssllraptiora made in examp)e a „ d , he of 

In most Government contracts reliability goals or require- allocating are as follows, 
ments are specified at the system level only. The apportioning (1) All parts are required for system success, 

of these goals to elements of the system is left to the contractor. (2) All parts fly the entire mission. 
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Thus, if the system reliability requirement is to be met, high 
failure contributors must be offset by low ones so that their 
average ‘ailure rate \ p s 2/10 -6 . 

Let us continue this example by further examining one 
specific part class. For MIL-HDBK-217E (ref. 4-2) values 
this flight failure rate \ p includes the K A value associated with 
each part class (i.e., 1.5 to 100 for resistors). Thus, the \ 
for the fixed-film resistors (MIL-R-22684) in the system 
becomes 


\ - ^dK a 



0.25/10 _6 


Now that we have determined the X D requirement, we are 
ready for the next step. 

A quick scan of the \ D values, extracted from MIL- 
HDBK-217E (ref. 4-2) and shown in table 4-3, for this type 
of fixed-film resistor reveals the part temperature and stress 
ratio combinations that provide \ D s 0.25/10 " 6 . The anti- 
cipated operating temperature, say 40 °C maximum, would 
further reduce the acceptable combinations, leading to the 
conclusion that this type of part must be derated 60 percent 
or more to meet the reliability apportionment. 

The second method of allocating failure rates is called the 
equal-risk method and can also be applied when allocating re- 
liability goals to several elements within a system (see fig. 4-5). 

The reliability assigned to each element is given by 


Re = ”y/R s 

where n is the number of elements. The same reliability goal 
is assigned to each element, hence, the name “equal risk.” 
Example 2: A fire control system computer is made up 
of 10 logic racks and has a reliability requirement of 0.999. 
Allocate a reliability goal to each of the logic racks. 



Figure 4-5.— System elements model. 



flrack = 'V 0.999 = (e-o.mijno = ^-o.ooi = 0.9999 

The part failure rates \ p of each rack can then be allocated 
as shown in example 1 : 

(1) All subelements operate for approximately the sa . 
period. 

(2) There is no significant difference in the failure rate 
complexity of the subelements. 

Many other methods of allocating reliability goals take into 
account operating time, complexity, cost, maintainability, the 
state of the art, and other factors. See references 4-2 to 4-12. 

Importance of Learning From Each Failure 

When a product fails, a valuable piece of information about 
this product has been generated. We now have the opportunity 
to learn how to improve the product if we take the right actions. 

Failures can be classified into categories: | 

(1) Catastrophic failures— for example, a shorted transistor 
or an open wire-wound resistor 

(2) Degradation failures— for example, change in the gain 

of a transistor or the value of a resistor j 

(3) Wearout failures — for example, the wear of brushes in 
an electric motor 

These three principal failure categories can be broken down 
further: 

(1) Independent failures— For example, a shorted capacitor 

in a radiofrequency amplifier has nothing to do with a low- | 

enr ssion cathode in a picture tube. i 

(2) Cascade failures— For example, the shorted capacitor 
in the radiofrequency amplifier causes excessive current to 
flow in its transistor and burns the collector beam lead open. 

(3) Common mode failures— For example, uncured resin 
is present in motors. 

By using these categories and a good failure reporting, J 

analysis, corrective action, and concurrence system, much can 
be learned from each failure. Failure analysis is required to 
determine what caused the part to fail. Corrective action 
ensures that something was done about the cause. Concurrence 
keeps management informed on what is being done to avoid 
another failure. These data enable all personnel involved to 
compare the part ratings with the use stresses and thus verify 1 

that the part is being used with a known margin. ! 

Failure Reporting, Analysis, Corrective [ 
Action, and Concurrence | 

A number of different methods can be used to record S 

reliability data for any given project. The Department of j 

Defense has standardized a method on DD form 787-1. A f 
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simple form that tells the whole story on one sheet of paper 
is NASA-C-8192 (fig. 4-6). The method that you use to 
record reliability data will have to fit your needs. Keep your 
form simple and easy to fill out, and get approval from 
management. 


Case Study— Achieving Launch Vehicle 
Reliability 

Design Challenge 

The launch vehicle studied requires the highest acceleration 
and velocity and the shortest reaction time of any developed. 
As such, the design challenges were formidable; typical in- 
flight environments include random vibration of 61 g’s rms up 
to 3 kHz, mechanical shock at 25 000 g’s peak (between 5 and 
10 kHz), linear acceleration well in excess of 100 g’s, acoustics 
of 150 dB, and aerodynamic heating up to 6200 # F. The devel- 
opment philosophy was tor a vehicle to be launched from a 
tactical silo with the initial design. Although many changes 
occurred during the 13 -year development, the first flight test 
vehicle was not greatly different from the 70 now deployed. 


Subsystem Description 

The vehicle is launched from an underground silo, which 
also serve* as a “storage container” during the multiyear 
design life. Adjacent to the silo and integral to it is a small 
compartment housing the ground support equipment. This 
equipment is used to conduct periodic tests of the vehicle 
electronics, to prepare the vehicle for launch, and to launch 
the vehicle. It also maintains the silo environment at 
80 ± 10 *F and 50 percent or less relative humidity. 

The vehicle is predominantly in a power-off storage mode 
when deployed in its silo. A periodic test of flight electronics 
is conducted automatically every 4 weeks. In a multiyear 
design life the flight electronics accumulate about 1 1 min of 
operating time and 43 830 hours of storage time. The ratio 
of storage time to operating time is nearly 240 000:1. 


Approach to Achieving Reliability Goals 

Reliability mathematical models were developed early in the 
research and development program. From these models it was 
apparent that the following parameters were the most important 
in achieving the reliability goals: 

(1) Electronic storage failure rate during a multiyear design 
life (i.e.. storage failures) 

(2) Percent testability of missile electronics (i.e., M1L- 
STD-471A, ref. 4-6) 

(3) Periodic test interval for missile electronics 

(4) Severity of in-flight environments (acceleration, shock, 
vibration, and aerodynamic heating) 



Launch and Flight Reliability 

The flight test program demonstrated the launch and flight 
reliability of the vehicle. The ultimate program success 
ratio of 91 percent exceeded the overall cvaiiability-reliability 
goal by a comfortable margin. 


Field Failure Problem 

Twenty-six guidance sections failed the platform caging test 
portion of the launch station periodic tests (LSPT’s). These 
failures resulted in a major alarm powerdown. An investigation 
was conducted. 

Description of launch station periodic tests .— The system 
test requirements at the site include a requirement for station 
periodic tests upon completion of cell or vehicle installation j 
and every 28 days thereafter. LSPT’s check the overall system 
performance to evaluate the readiness of a cell. During an 
LSPT the software initiates a test of the vehicle and ground 
equipment, data processing system, and radar interfaces. Any 
nonconformance during an LSPT is logged by the data 
processor and printed out, and the time from initiation of LSPT 
to failure is recorded During an LSPT the platform spin motor 
is spun up and held at speed for approximately 10 sec. After 
this the system is returned to normal. 

An LSPT consists of two phases: 

(1) Spinup— a powerup phase to spin the gyros, align the j 
platform, verify platform null, and check airborne power 
supply operation 

(2) A detailed test of airborne electronics in the radio- j 

frequency test phase j 

Initial failure occurrence . -Cell 3 on remote farm 1 (RIC3) 
experienced an LSPT failure (a major alarm powerdown) 

5.936 sec after prep order,” the command '.o get the vehicle 
ready to launch. The failure did not repeat during four 
subsequent LSPT’s. R1C3 had previously passed three 
scheduled LSPT s before failure. A total of four cells on 
remote farms 1 and 2 had experienced similar failures. Two 
of the failures occurred at 5.360 sec (an inverter test to 
determine if ac power is available). Two occurred at 5.936 
sec (caging test to determine if the platform is nulled to the 
reference position; see fig. 4-7). 

Replacement of failed guidance and control sections (G&C) 

28, 102, and 86 led to successful LSPT’s. G&C 99, which 
failed only once during in-cdl testing, was left on line. G&C’s 
28. 102, and 86 were returned to Martin Marietta, Orlando, 
for analysis of the presumed failed condition. 

Failure verification and troubleshooting . — A test plan was 
generated that permitted testing of the failed G&C’s in a 
horizontal marriage test and a G&C test to maximize the 
probability of duplicating the field failures. Test results 
confirmed site failures for both the caging null and the inverter 
null during a horizontal marriage tesi on G&C 102, a GAC 
level test on G&C’s 28 and 86, and an autopilot level test on 
G&C 102. G&C 102 failed caging null four times and inverter 
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Figure 4-6.— Concluded, 


null once at horizontal marriage. Evaluation of the inverter 
null failure revealed that a high caging amplifier output caused 
the launch sequencer level detector to become offset during 
inverter monitoring, resulting in the major alarm even though 
the autopilot inverter voltage was normal. Launch sequencer 
offset may or may not occur with an uncaged platform 
depending on the amplitude of the caging amplifier output 
when the inverter voltage is monitored. Therefore, both the 
inverter null and the caging null LSPT failures at site were 
due to failure of the platform to cage. 

An autopilot acceptance test tool was modified to permit 
monitoring of the platform spin motor voltage (800 Hz, 8 V, 
3 0) and the spin motor rotation detector (SMRD). During 
a spinup test on autopilot 69 (G&C 102), recordings indicated 
sustained caging oscillation. The SMRD showed no evidence 
of spin motor operation even though ail autopilot voltages were 
correct, including the spin motor excitation voltage at the 
platform terminals. Further verification was obtained by 
listening for characteristic motor noises with a stethoscope. 

G&C 86 failed the G&C level test due to caging null and 
inverter null alarms. Then, 3.5 sec into the third run the caging 
loop stopped oscillating, but the platform did not cage in time 
to pass the test. The next run met all G&C test requirements. 
It appeared obvious that the spin motor started spinning in the 
middle of the run. 

G&C 28 failed one run of the G&C level test; however, 
it met all requirements in the autopilot level test. This means 
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Figure 4-7.— System spinup tests. (Gate times are within ±50 ms of that 
shown because of data processor tolerances.) 


that the spin successfully met its acceptance test procedure 
requirements. A hesitation was noted during two of the seven 
spinup tests conducted. Platform 127 was heated to normal 
on the gyro test set. Its resistances were checked and found 
to meet specification requirements. No attempt was made to 
start platform )27\ spin motor at platform level. Both units 
were hand-carried to the subcontractor for failure analysis. 
The subcontractor was familiar with the construction of the 
platform and had the facilities to disassemble the platform 
without disturbing the apparently intermittent failure condition. 

Verification test conclusions .— Verification tests isolated 
the site LSPT failures to a failure cf the platform spin motor 
to spin up, thereby causing major alarms at the inverter null 
or caging null gate. During testing, three of the first four failed 
platforms caged upon repeated application of voltage. Once 
the platform caged, the platform, autopilot, and G&C met all 
system te st requirements. On the basis of these results, it was 
decided to repeat LSPT’s up to 10 times after a site failure 
before removing the G&C, If the LSPTs were successful, the 
G&C would be left on line. 

Measurements at platform level indicated the problem was 
internal to the platform and that all resistances and the plat- 
form temperature were correct. Subcontractor representatives 
reviewed the test results and concurred that the problem was 
internal to the platform. 

Mechanical Tests 

The spin motor breakaway torque was measured with a 
gram gage on platform 127 and was found to be normal 
(750 dyne cm). Dynamometer tests were performed on both 
platforms. The dynamometer is an instrument that mtasures 
rotation torque by slowly rotating .he rotor of the spin notor 
while recording the stator rotationa' torque. The dynamometer 
is use 1 during initial builds to establish the spin motor bearing 
preload (torque). The spin motor generates approximately 4000 
dyne cm of starting torque with normal excitation voltage; 800 
dyne cm of this torque is used to overcome the inertia and 
frictional torque of the motor. 

Platform 140 was tested on the dynamometer and produced 
the torque peaks of 3400 and 3100 dyne cm shown in ilgure 4-8. 
The torque peaks were three revolutions apart. This is four 
times the normal running torque level for a new spin motor 
and about four times the torque level for this spin motor for 
the rest of its run. The torque increase lasted for about one- 
half of a revolution and repeated within three revolutions. The 
spin motor bearings were cleaned and reassembled. Two large 
torque spikes of approximately 3000 dyne cm were observed 
on the first revolution. A 2200-dyne cm torque hump, one 
revolution in duration, was centered at the beginning of the 
second revolution. From these results it was concluded that 
something in the spin motor bearing was causing an abnormal 
frictional load in the bearing. This result isolated the problem 
to the spin motor bearing area and eliminated the motor 
electrical characteristics as being a contributor. 
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Figure 4-8. —Plat form dynamometer torque test. 

Runup and Rundown Tests 

A series of tests were performed on spin motors 96 and 140 
to determine the effect of motor running time on spin motor 
start and running torque. Figure 4-9 shows the change in 
rundown time with change in motor run time. 


Summary of Case Study 

Field problem cause. — The 26 LSPT failures at the site were 
caused by the failure of the G&C platform spin motors to spin 
up within 6 see after the command to get the vehicle ready 
for launch. It was determined that the spin motors did not start 
with normal application of voltage. A polymer film had formed 
on the bearing surfaces during testing at 175 °F and caused 
the bails to stick to the outer race. This film was identified 
as from the alkyl phenol and alkyl benzene families. Its source 
was determined to be uncured resins from the bearing retainer. 

Polymer film.— A film approximately 900 A thick had 
formed on the metal surfaces of the bearings of failed spin 
motors. The amount of material generated was - 10~ 7 g/ball. 
To put this number in proper perspective, 2x 10 " 4 g of oil 
is put on the bearing race during initial build, and 2 x 10 “ 3 g 
of oil is impregnated in the bearing retainer. 

Alkyl phenol/alkyl benzene is a generic identification of a 
family of organic compounds. Further analysis identifies the 
major compounds in the family as phenol and methyl phenol 
(alkyl phenols) and toluene, xylene, and benzene (alkyl 
benzenes). A phenolic polymeric film would hr /e the gummy, 
adhesive, and insolubility properties detected in the analysis. 
There is little doubt the gummy film detected was a phenol- 
based material. 


Source of phenol .— Phenols are used in three areas of the 
spin motor. A phenolic adhesive bonds the stator laminations 
together and bonds the hysteresis ring to the rotor. The bonding 
processes adequately cure the phenol to the point where un- 
cured phenols would not be present. Also, the stator laminations 
arc coated with epoxy after bonding. The remaining source 
is the paper phenolic retainer, which serves as a spacer and 
a lubrication source for the spin motor bearings. Mass spectral 
analysis of the retainers yielded spectra essentially identical 
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to the spectrum of the coating on the failed bearings. The 
conclusion of this analysis is that the source of the phenolic 
is uncured phenolic resins or resin compounds in the retainer. 

Retainer processing.— The retainer material is manufactured 
by a vendor to military specifications and screened to tighter 
vehicle requirements for specific gravity. There is no specific 
requirement concerning uncured resins in the retainer material. 
The vendor estimated an upper limit of 1 percent of uncured 
resin in the retainer raw material. One percent would provide 
3 X 1C - g of uncured resins, more than sufficient to cause 
the spin motor problem. 

The finished retainer material is cleaned by an extraction 
process with benzene or hexane. This process does not remove 
a significant amount of uncured resins. Therefore, if uncured 
resins survive the vendor processing, they will remain in the 
uncured state in the installed retainers. 




resins are transferred from the retainer to the bearing surfaces 
through the natural lubricating process of the retainer. Running 
the spin motors generates centrifugal forces that sling the 
excess oil off the rotating surfaces, leaving a thin film of oil. 
The force of gravity during subsequent storage of the motor 
causes the already thin film to become thinner on the top 
surfaces and thicker on the lower surfaces. This redistribution 
process involves only the oil and leaves more viscous con- 
taminants in place. Subsequent running of the motor will cause 
replacement of oil on the oil-free surfaces. The source of the 
replacement oil is the retainer capillaries. This replacement 
process will cause the oil to bring any uncured phenolics to 
the surface of the retainer. The metal surfaces will then become 
lubricated with oil containing a small percentage of uncured 
resins. Subsequent storage cycles and running will continue 
this redistribution process, steadily increasing the phenolic 
concentration. Exposure to a temperature of 175 C F and ex 
tended operational maintenance gradually cures these phenolics 
in two stages. Initially, ? highly viscous gummy residue is 
formed; finally, a hard insoluble polymer film is formed on the 
metal surfaces. The film forms a bond between the balls and 
the races. The crating builds up to the point where the spin motor 
torque cannot overcome the bond at initial power application. 
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Extent of problem .— Analysis of failed and unfailed field 
units proved that not all platforms are susceptible to this failure. 
Obviously, a high percentage are susceptible, since 26 failures 
have been experienced. It is likely that many unfailed platform? 
contain some small percentage of uncured resins. 

The significantly higher failure rate in the units with higher 
serial numbers points to a process (or common) failure mode. 
AH evidence points to lot-to-lot variations in the amount of 
uncured resins preser.; in the retainer raw material. Traceability 
from retainer lot to individual platform spin motor was not 
possible in this case, but such records should be available. The 
26 units that have failed and the failure rate at the 14-day 
interval bound the total platform failure rate. The number of 
spares available is adequate to meet system life and reliability 
requirements. 

Site reliability . —The site system reliability goal allows 
approximately two G&C failures per month for any cause. 
Analysis of test data indicates the goal can be achieved at either 
a 7-day test interval (0.8 failure/month) or a 14-day test interval 
(1.5 failures/month). It cannot be achieved at a 21-day interval 
(7.7 failures/month) or a 28-day interval (8.6 failures/month). 
Even though at least 74 percent of the site failures were 
restarted, a limited number of spare G&C’s are available. 

Tests at the site revealed that most failed spin motors can 
be restarted within 10 power applications and, once started, 
will perform properly. The site procedure was revised to leave 
any failed G&Cs that restart within 10 attempts on line. 
Platforms that did not start within 10 attempts were returned 
to the contractor and were restarted by repetitive application 
of overvoltage or reverse voltage up to the motor saturation 
limit. These data support the conclusion that the failure mode 
was the formation of a film bond on the race and that increasing 
the inverter output voltage to the motor saturation limit would 
not eliminate the problem. 

Current site operating procedures provide a 14-day LSPT 
interval with a 10-min run time. This enables the G&C failure 
rate to meet system reliability goals. The vehicle site is 
currently being deactivated. If reactivation should be required, 
the repair of all defective or support platforms should be 
included as part of that effort. 

Concluding Remarks 

Now that you have completed chapter 4, several concepts 
should be clear. 

(1) The failure rate of complex equipment is usually con- 
sidered to be a constant. 


(2) Most failures are random, with repetitive failures 
representing a small portion of unreliability. 

(3) The rate at which failures occur depends upon 

(a) The acceptance criteria, which determine how effec- 
tively potential failures are detected 

(b) All applied stresses, including electrical, mechanical, 
and environmental. (As these stresses increase, the failure 
rate usually increases.) 

(4) Published failure rate data represent the potential failures 
expected of a part. The rate at which these failures are observed 
depends on the applied electrical stresses (the stress ratio) and 
the mechanical stresses (the K A factor). 

(5) In general, failure rate predictions are best applied on 
a relative basis. 

(6) Failure rate data can be used to provide reliability 
criteria to be traded off with other performance parameters 
or physical configurations. 

(7) The reliability of a device can be increased only if the 
device's failure mechanisms and their activation causes are 
understood. 

In addition, you should be able to use failure rate data to 
predict the failure rate expected of a design, and consequently, 
to calculate the first term, P c , of inherent reliability. Finally, 
you should be able to allocate failure rate requirements to parts 
after having been given a reliability goal for a system or the 
elements of a system. 
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Reliability Training 1 

la. Using the failure rate data in table 4-4 (on p. 51), calculate the flight failure rate for a launch vehicle 
electronic subsystem consisting of the following parts (assume K A - 1000): 


Component 

Number of 
parts, 

N 

Resistor, G657I09/I0 

5 

Resistor, variable. II 176416 

1 

Capacitor. G657II3 

3 

Diode, G657092 

3 

Transistor. II 176056 

4 

Integrated circuit, analog, 11177686 

I 


A. 195 failures per 10 9 hours 

B. 195 000 failures per 10 9 hours 

C. 195 000 failures per 10 6 hours 

lb. Assume the flight failure rate for this circuit is 500 000 failures per I0 9 hours. Calculate the 
reliability of the circuit for a 0.01-hour flight. 

A. 0.9995 B. 0.99995 C. 0.999995 

2. The a posteriori flight failure rate of a launch vehicle is 440 000 failures per 10 9 hours. 

a. If the storage failure rate is 0.3 of the operating rate, how long can the vehicle be stored with a 
90.4 percent probability of no failures? 

A. 30 days B. 40 days C. 50 days 

b. After 1450 hours (2 months) in storage the vehicle is removed and checked out electronically. If 
the vehicle passes its electronic checkout and the checkout equipment can detect only 80 percent 
of the possible failures, what is the probability that the vehicle is good? (Ignore test time.) 

A. 0.962 B. 0.858 C. 0.946 

3. A subassembly in a piece of ground support equipment has a reliability requirement of 0.995. Preliminary 
estimates suggest that the subassembly will contain 300 parts and operate for 200 hours. What is the 
average part failure rate required to meet the reliability goal? 

A. 25X10" 6 B. 16 667x10"’ C. 83X10" 9 


4. A piece of ground support equipment has a reliability goal of 0.9936. It contains four subassemblies 
of approximately equal risk. 

a. What is the allocated reliability goal of each of the four subassemblies? 

A. 0.99984 B. 0.9984 C. 0.9884 


b. Allocating further into subassembly 1. Assume the goal is 0.998. Solve for the average part failure 
rate given the following: 


Estimated parts count: 100 
Estimated operating time: 10 hours 

A. 20 000X 10 “ 9 B. 2000 x 10 " 9 C. 200x10"’ 


'Answers art given al the end of ihis manual. 







... j !i yi uijiii i . mm JWSJPMllPP 


TABLE 4-4.— SELECTED LISTING— APPROVED ELECTRONIC 
FAILURE RATES FOR LAUNCH VEHICLE APPLICATION* 


Pin number 

Pan 

Operating mode 

b Nonoperating 
mode* 

Failure rate, failures/ 10* hr 

Integrated circuits j 

1 1 177680/81/82/83/84/85 

Digital 


10 


3 

I 1177686 

Analog 


30 

10 

Transistors 


6SS7IS5 

Double switch 


10 


3 

6557318/19 

Medium-power switch 

20 



6557046 

PNP type of transistor 





11176911 

Medhim-power switch 





11176056 

High-speed switch 





11177685 

Field -effect transistor 





6310038 

2N520I 

10 



6557072 

2N918 (unmatched) 

L 50 


5 

Diodes 


6557061 

Rectifier and logic (5 V) 

10 


3 

6557092 

Rectifier and logic (30 V) 


5 



6557123 

Rectifier and logic <50 V) 





6557125 

Rectifier and logic (600 V) 





1 1 1769 12 

Rectifier and logic (400 V) 

1 




Resistors 


6557018 

2.5-W wire wound 


2 


! 

6557015 

1/8- W wirewound 


3 

2 


6557016/17 

1- and 2-W wirewound 


2 


.5 

6557030 

1/10-W fixed film 


l ! 


.5 

6557031 

6-W wirewound 


5 


.5 

6557109/10 

I/4-W fixed composition 


1 


.2 

6557329 

1/8- W fixed film 


1 


.3 

11176416 

1-W variable metal film 

50 

10.3 

Capacitors 


G657020/2I/22 

Fixed glass 

0.1 

0 1 

G6571 13/173 

Fixed ceramic 

5 


1 


G657II4 

Fixed ceramic 

10 I 

1 


G6571 19/120 

Solid tantalum 

2 


I 


G657202 

Precision, fixed ceramic 

50 

3 


Relays 


1 1 176326/453 

DPDT armature 

100 

20 


Transformers (RF) 


1 1301034/35/43/49 


10 


5 


11301064 


1 


5 



RF coil 


G657 140/41 


3 

2 

G657 178/81 


10 

2 

RF filter 


G657189 


50 

5 


•Cun.-nt failure rate data arc available fmni »wo tourers ir«fv 4- 1 and 4-4), [ 

^AppUe* w all ilaeh numbers of pans shown (Wotm caw shown.) j 
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Chapter 5 


i Applying Probability Density Functions 


The inherent reliability of equipment is defined in chapter 3 as 

l.'\ Ri — e~ u PJP w 

■J : \ ; where 

Rj probability of no failures 

L e~ h probability of no catastrophic part failures 
P, probability of no tolerance failures 

P w probability of no wearout failures 

Before discussing the P, and P w terms in the next chapter, it 
. is necessary to develop an understanding of probability density 
functions and cumulative probability functions. These concepts 
I form another part of probability theory not discussed in 
chapter 2. First, in this chapter the theory of density and 
cumulative functions is discussed in general; then the normal, 
or Gaussian, distribution is discussed in detail. This normal 
distribution is used extensively later in the manual. 


Probability Density Functions 

If a chance variable .v can take on values only within some 
interval, say between a and b, the proba jility density function 
p(.x) of that variable has the property that (ref. 5-1) 

( p(x) dx = 1 

i 

l 

'{ In other words the area under the curvep(jr) is equal to unity. 

\ This is shown in figure 5-1. 

y In the language of probability, the probability of x being 
within the interval (a,b) is given by 


In a similar fashion we can find the probability of x being 
within any other interval, say between c and d. from 


P(c £x< 



dx 


This is shown in figure 5-2. 

Example /: Suppose we were to perform an experiment in 
which we measured the height of oak trees in a 1-acre woods. 
The result, if our measuring accuracy is ±5 feet, might look 
like the histogram shown in figure 5-3. 

The value at the top of each histogram cell (or bar) indicafes 
the number of trees observed to have a he: »ht within the 
boundaries of that cell. For example, 19 trees had a height 
between 6 and 10 feet, 17 trees had a height between 10 and 
20 feet, etc. The figure shows that 100 trees were observed. 

Now let us calculate values for the ordinate of the histogram 
so that the area under the histogram equals unity. Then, we 
will establish a probability density function for the tree heights. 
Since we observed 100 trees, it should be apparent that if the 
calculated ordinate of a cell times the width of the cell (the 
cell area) yields the percentage of 100 trees in that cell, the 
sum of the percentage in all cells will have to equal 100 
percent. Of, if the percentages are expressed as decimal 
fractions, their sum will equal 1 , which will be the total area 
under the histogram. Therefore, 


Ordinate of cell = 


Percent of trees in cell 
Width of cell 


For the cell 0 to 10 feet, which has 19 percent of the trees in it. 


P(a s x £ b) = f p(x) dx = 1 


19 

Cell ordinate = — x — = 0.019 
100 10 


In other words the probability that x lies between a and b is 
I . This should be clear, since .v can take only values between 
a and b. 


As a check, we can see that 

Cell ordinate - 0.019 x Cell width (10) = 0.19, or 19 percent 
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The area under the curve is (ref. 5-2) 

>100 .100 

Area = J o p(x) dx = ) (-0.0002* + 0.02) dx 

= -il + 0.02rr=-^ + 0.02(.00) 

I0 4 'o 10 4 ’ 

I0 4 „ 

= t + 2= -1 +2= 1 

10 4 

This agrees with our requirement that the area under a probability 
density function equal unity. 


Application of Density Functions 

Now let us see how we can apply the density function to the 
tree data. To find the percentage of trees between 60 and 80 
feet high, solve for 


(.80 .80 

P ( 60 s * £ 80) = J w p(x) dx = (-0.0002* + 0.02) dx 


,80 


= -— + 0.02* — ( 80 2 - 60 2 ) + 0.02(80 - 60) 


'60 10 


= - — (2800) + 0.4 = -0.28 + 0.4 


= 0.12, or 12 percent 


Figure 5-3 shows that this answer is correct, since 12/100 trees 
were observed to have a height between 60 and 80 feet. 

Another way to look at this example is that there is only a 
12-percent chance that a tree picked at random from the 1-acre 
area would have a height between 60 and 80 feet. In a similar 
fashion we can calculate the probability that a tree would have 
any range of heights within the boundary of 0 to 100 feet. 

In the tree example, we were able to measure the trees in a 
particular part of the woods and to obtain a height density function 
for those trees. But what do we do if we are interested in a 
different area of woods and for some reason we are not able 
to go out and measure the trees? We would probably assume 
that the acre we measured was representative of all other acres 
in the same woods. If we accept this assumption, we could then 
use our experience (the established density function) to predict 
the distribution of tree heights in an unmeasured acre. And this 
is exactly what is done in industry. 

As you can see. if we know what the density functions are 
for such things as failure rates, operating temperatures, and 
missile accuracy, it is easy to determine the probability of meeting 







Target miss distance, ft 

Figure 5-5.— Probability density function for missile target miss distance. 


a failure rate requirement for equipment (such as a missile) 
specified to operate in some temperature range with a required 
accuracy. 

Example 2: Suppose a missile has a maximum target miss 
distance requirement of 90 feet and that after several hundred 
firings the probability density function for miss distance is 

p(x) — -0.0002* + 0.02 where 0 < x < 100 


which is the same as the/^x) for the tree example. This is shown 
in figure 5-5. 

To predict the probability that the next missile fired will miss 
the target by more than 90 feet, solve for 


.100 

P(90 <; * < 100) = ) (-0.0002* + 0.02) dx 


= - A + 0 . 02 * 

10 4 


100 

90 


= (100 2 -90 2 ) +0.02(100-90) 


1900 

= -_r + 0 - 02 <‘°) 


= - 0. 19 + 0.2 = 0.01, or 1 percent 

In other words there is a 99-percent chance that the missile 
will hit within 90 feet of the target and a 1 -percent chance 
that it will not. This is shown as the shaded area under the 
density function in figure 5-5. 


Cumulative Probability Distribution 

Another practical tool in probability calculation is the 
cumulative probability distribution, denoted by F(x) (ref. 5-3). 
An F(x) curve for the tree example in the preceding section 
is shown in figure 5-6. The curve represents the cumulative 
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Figure 5-6. -Cumulative probability function for tree heights. 

area under the probability density function p(x). The ordinates 
of the curve were calculated as shown in table 5-2. 

The cumulative curve can be used to solve the same prob- 
lems as the density curve. 

Example 3: Referring again to example 1 , suppose we want 
to know the probability that a particular tree selected at random 
from the woods will have a height between 30 and 50 feet. 
Solution 3 A: Using the density function for tree height. 


P(30 < .v < 50) = J (-0.0002* + 0.02) dx 


,50 


= - —7 + 0.02v 
10 4 '30 


TABLE 5-2.— ORDINATES FOR CUMULATIVE 
DISTRIBUTION OF TREE DATA 


Tree height, 
ft 

Area under 
p(x) curve 

Ordinate of p(x ) curve 
(cumuiati\e area) 

0-10 

0.19 

0.19 

10-20 

.17 

.36 

20-30 

.15 

.51 

30-40 

.13 

.64 

40-50 

! .11 

.75 

50-60 

.09 

.84 

60-70 

.07 

.91 

70-80 

.05 

.96 

80-90 

.03 

.99 

90-100 

.01 

1.00 


probability function F(x) is found from 


F(x) = j p(x) dx 
and 

!„/>(•*) dx = F(b) - F(a) 
For the tree example 


F(x) = j (-0 


0002* + 0.02) dx = - il + 0.02* 


Consequently, we can find the probability of a variable* being 
within some interval by using the cumulative function F(x) 
even though the cumulative graph is not available. 

Example 4: What is the probability that a tree selected at 
random will have a height less than 20 feet? 

Solution 4: 


P(0 S * < 20) = J q p( x ) dx = F( 20) - F( 0) 


.20 


= 7 + 0 . 02 * 

10 4 


1600 

10 4 


+ 0.40 



+ 0 . 02 ( 20 ) 


-0 


- 0.16 + 0.40 = 0.24, or 24 percent 

Solution 3B : Using the cumulative curve shown in figure 5-5, 
P(30 s * s 50) = F(50) - F(30) = 0.75 - 0.51 

= 0.24, or 24 percent 

which agrees with solution 3A. 

Note that in working out solution 3A the next-to-last step 
(0.75 - 0.51) is the same as the next-to-last step of solution 
3B. The reason for this is that the equation of the cumulative 


— 0.04 + 0.4 — 0.36, or 36 percent 

which agrees with a graphical solution. 

Some general rules for the use of the cumulative function 
F(x) are 

(1) P(xsa) -F(a) 

(2) P(x a a) = 1 - F(a ) 

(3) P(aaxizb) * F(b) - F(a) 

Example 5: Suppose we would like t ' '.now the probability 
of equipment seeing tropic zone temperatures above !20 *F 
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f.; during operation because ; ; above 120 *u «/e have co add a 
costly air-conditioning system to the equipment. If we could 
obtain the temperature data, we might find that the cumulative 
distribution for tropic zone temperatures would be that shown 
in figure 5-7. 

Solution 5: From the curve the probability of observing a 
- j temperature at or above 120 °F is given by 

^(temp > 120 °F) = 1 — F( 120 °F) = 1 - 0.97 



Figure 5-8.— Histogram and density function for heights of children. 


for the heights of the children. Such a curve (sometimes called 
a bell curve) is the shape of the normal distribution. We say 
that the children’s heights are distributed normally. 

Normal Density Function 

The equation for the density function p(x) of the normal 
distribution is 


PM = — i = . e .-('-.v) 2 /2a-’ 
ffV 27T 


This curve is shown in figure 5-9. The function p(x) has two 
parameters. The first is the mean x calculated from 


= 0.3, or 3 percent 

With only a 3-percent chance of temperatures abr 120 °F. 
! we P rob ably would decide against air-conditiopi g (all other 
parameters, such as failure rate, being equal). 


x = - Lx, 

» i= t 


! Normal Distribution 

i ! 

( ! 0ne . of ,he most frequently used density functions in 

r | reliability engineering is the normal , or Gaussian, distribution. 

, , A more descriptive name, however, is the norma curve of 

r I error because it represents the distribution of errors observed 
>.A from repeated measurements of an object or some physical 
r l phenomenon (ref. 5-4). 

i Example 6\ Assume that we need to measure the heights 
■" [ of eighth grade children. A histogram of the children’s heights 
' ! wou,d resemble the curve in figure 5-8. If, as in our tree 

i example, we calculate an ordinate for the histogram so that 

r | ‘he area under the histogram equals unity and then connect 

i the midpoints of each cell, we obtain a smooth curve as shown 
L'- l figure 5_8, This curve represents the density function 
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Figure 5-9.— Normal density function. 


where 


TABLE 5-3. -AREAS BETWEEN -z AND z 


n total number of measurements or observations 
.\j value of i th measurement 

The mean, therefore, is the arithmetic average of the meas- 
urements. From example 6 we would add up ail the heights 
observed and then divide by the number of children measured 
to obtain a mean or average height. The mean of all the 
children’s heights from the data in figure 5-8 is 5.3 feet. 

The second parameter of p(x) is the standard deviation a 
calculated from 


Z 

Area under curve 

Probability, 
< .r s; 

1 

0.683 

P(-\o < x <. la) 

2 

.9545 

P(-2a < x < 2a) 

3 

.9973 

P( s. x < 3a) 

4 

.999937 

P(-4o < x < 4a) 

5 

.999999426 

P(-5a £ x < 5a) 

6 

.99999999803 

P( -6a < x < 6a) 

7 

.999999999992 

P{ -la < x < 7a) 


<,= £ { x i ~ x)2 

1=1 a -1 

where 

x mean of measurements 
Xj value of i th measurement 
n total number of measurements 

Note that n - 1 is used in the equation in order to give an 
unbiased sampling distribution. In the general definition of a, 
n instead of w — 1 would be used. 

The standard deviation is the square root of the variance, 
which is denoted by a 2 . The magnitude of the variance, as 
well as the standard deviation, indicates how far all the 
measurements deviate from the mean. The standard deviation 
of the children’s height data, for example, is approximately 
0.3 foot. If the range of heights observed had been from 5 
to 5.6 feet, the standard deviation would have been approx- 
imately 0.1 foot. And with this standard deviation the 
distribution would look squeezed together as shown by the 
dashed curve in figure 5-8. However, the area under the 
dashed curve would still equal the area under the solid curve. 

Properties of Normal Distribution 

The normal density function is a continuous distribution from 
-oo to oo. It is symmetrical about the mean and has an area 
equal to unity as required for probability density functions. 
For the normal distribution the standard deviation is the 
distance on the abscissa from the mean x to the intercept on 
the abscissa of a line drawn perpendicular to the abscissa 
through the point of intlection on the curve. This is shown 
in figure 5-9. It is also shown that equal increments of the 
standard deviation can be laid out to the left (-) and the right 
(+) of the mean x . 

As you will recall, in determining probabilities from a 
density function, we need to calculate the area under the curve 
p(x). When using the normal density function, it is common 
practice to relate areas to the standard deviation. In general, 
for the area under the curve between the values of z and - 2 , 


standard deviations can be found from 

p[-z < x < z] = Area — f — -= c" ,/2(:) “ dz 

cfV2tt 

The areas for various values of z are shown in table 5-3. This 
table says that the area under the normal curve between la 
and - 1 a is 0.683, or 68.3 percent; the area under the normal 
curve between 2 a and -2a is 0.9545, or 95.45 percent, etc. 

Example 7: The term “3a limit” refers to the area under the 
normal curve between 3a and -3a, which is 0.9973, or 99.73 
percent, as shown in table 5-3. Therefore, if a power supply 
output is defined as 28 ± 3 V and the ± 3 V represents a 3a limit, 
99.73 percent of all such power supplies will have an output 
between 25 and 31 V. The percentage of supplies having an 
output greater than 31 V and less than 25 V will be 1 - 0.9973 
= 0.0027, or 0.27 percent. This is shown in figure 5-10. 

Up to now we have been working with areas under the 
normal density function between integers of a, that is, 1, 2, 
3, etc. In practice, however, we are usually interested in the 
area between decimal fractions of a, those being 1 . 1 , 2.3, etc. 
We have also been using z to represent the number of standard 
deviations that a particular limit value is from the mean. For 
instance, in the power supply example 25 V was given as being 
three standard deviations from the mean of 28 V. It is better 
when working in decimal fractions of a to let z = (x - x )/a, 
where x - x is the distance from the mean x to the limit value 
and a is the standard deviation. Going back to the supply 
example, our lower limit was 25 V. This was 3 V from the 
mean of 28 V, and the standard deviation was 1 V; therefore, 
2 = (25 - 28)/ 1 = -3. 

Symmetrical Two-Limit Problems 

In this discussion the term “symmetrical two-limit problems” 
refers to the area under the density function at equal values 
of z from both sides of the mean. The power supply example 
was of this type, since we were concerned with the area 
between -3a and 3a ft , i the mean*. To work these problems 
when 2 is a decimal fra ;tion, we use tables of areas in the two 
tails of the normal curve. 
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TABLE 5-4.— AREAS IN TWO TAILS OF NORMAL CURVE AT SELECTED VALUES OF c 

[From reference 5-1.] 



0.01 


0.02 


0.03 


0.04 


0 1.0000 

.1 .9203 

.2 .8415 

.3 .7642 

.4 .6892 


0.9920 

.9124 

.8337 

.7566 

.6818 


0.9840 

.9045 

.8259 

.7490 

.6745 


0.9761 

.8966 

.818! 

.7414 

.6672 


0.9681 

.8887 

.8103 

.7339 

.6599 


.6171 

.5485 

.4839 

.4237 

.3681 


.6101 

.5419 

4777 

.4179 

.3628 


.6031 

.5353 

.4715 

.4122 

.3576 


.5961 

.5287 

.4654 

.4065 

.3524 


.5892 

.5222 

.4593 

.4009 

.3472 


0.05 


0.9601 

.8808 

.8026 

.7263 

.6527 

.5823 

.5157 

.4533 

.3953 

.3421 


0.06 


0.9522 

.8729 

.7949 

.7188 

.6455 

.5755 

.5093 

.4473 

.3898 

.3371 


0.07 


0.9442 

.8650 

.7872 

.7114 

.6384 

.5687 

.5029 

.4,13 

.3843 

.3320 


0.08 


0.9362 

.8572 

.7795 

.7039 

.6312 

.5619 

.4965 

.4354 

.3789 

.3271 


0.09 


0.9283 

.8493 

.7718 

.6965 

.6241 

.5552 

.4902 

.4295 

.3735 

.3222 


1.0 .3173 

1.1 .2713 

1.2 .2301 

1.3 .1936 

<.4 .1615 


.3125 

.2670 

.2263 

.1902 

.1585 


.3077 

.2627 

.2225 

.1868 

.1556 


.3030 

.2585 

.2187 

.1835 

.1527 


.2983 

.2543 

.2150 

.1802 

.1499 


.2937 

.2501 

.2113 

.1770 

.1471 


.2891 

.2460 

.2077 

.1738 

.1443 


.2846 

.2420 

.2041 

.1707 

.1416 


.2801 

.2380 

.2005 

.1676 

.1389 


.2757 

.2340 

.1971 

.1645 

.1362 


1.5 .1336 

1.6 .1096 

1.7 .0891 

1.8 .0719 

1.9 .0574 


.1310 

.1074 

.0873 

.0703 

.0561 


.1285 

.1052 

.0854 

.0688 

.0549 


.1260 

.1031 

.0836 

.0672 

.0536 


.1236 

.1010 

.0819 

.0658 

.0524 


.1211 

.0989 

.0801 

.0643 

.0512 


.1188 

.0969 

.0784 

.0629 

.0500 


.1164 

.0949 

.0767 

.0615 

.0488 


.1141 

.0930 

.0751 

.0601 

.0477 


.1118 

.0910 

.0735 

.0588 

.0466 


2.0 .0455 

2.1 .0357 

2.2 .0278 

2.3 .0214 

2.4 .0164 


.0444 

.0349 

.0271 

.0209 

.0160 


.0434 

.0340 

.0264 

.0203 

.0155 


.0424 

.0332 

.0257 

.0198 

.0151 


.0414 

.0324 

.0251 

.0193 

.0147 


.0404 

.0316 

.0244 

.0188 

.0143 


.0394 

.0308 

.0233 

.0183 

.0139 


.0385 

.0300 

.0232 

.0178 

.0135 


.0375 

.0293 

.0226 

.0173 

.0131 


.0366 

.0285 

.0220 

.0168 

.0128 


2.5 

.0124 

.0121 

2.6 

.00932 

.00905 

2.7 

.00693 

.00673 

2.8 

.00511 

.00495 

2.9 

.00373 

.00361 


.0117 

.00879 

.00653 

.00480 

.00350 


1 


.0114 

.00854 

.00633 

.00465 

.00339 


.0111 

.00829 

.00614 

.00451 

.00328 


.0108 

.00805 

00596 

.00437 

.00318 


.0105 

.00781 

.00578 

.00424 

.00308 


.0102 

.00759 

.00561 

.00410 

.00298 


.00988 

.00736 

.00544 

.00398 

.00288 


.00960 

.00715 

.00527 

.00385 

.00279 


0 


3 0.00270 

4 .0*633 

5 . 0*573 

6 .0«I97 


0.1 


0.00194 

.0*413 

.0*340 

.0*106 


0.2 


0.3 


0.4 


0.5 


0.6 


0.7 


0.8 


0.9 


0.00137 

.0*267 

.0*199 

.0*565 


0.0*967 

.0*171 

.0*116 

.0*298 


0.*674 

.0*108 

. 0 7 666 

.0*155 


0,0*465 

.0*680 

.0 7 380 

.0 ,0 803 


0.0*318 
.0*422 
.0 7 2 14 
.01041 1 


0.0*216 

.0*260 

. 0 7 120 

.010208 


0.0*145 

.0*159 

.0*663 

.0<0|05 


0.0*962 

.0*958 

.0*364 

.0 M 520 


Table 5-4 shows tabulated areas in two tails of the normal 
curve for selected values of z from the mean Jr. For example, 
when z = 3.0, the table shows that 0.00270 of the total area 
lies in the two tails of the curve below —3 o and above 3o. 
Because the curve is symmetrical, 0.00135 of the area will 
lie to the left of -3a and 0.00135 »o the right of 3a. Note 
that this agrees with Figure 5-10 for the power supply example. 

Example 8 (v ring table 5-4): Suppose that a circuit design 
requires that the gain 0 of a transistor be no less than 30 and 
no greater uian 180. The mean Jf of the 0 density function of 
a particular transistor is 105 with a standard deviation of 32. 


What percentage of the transistors will have a 0 within the 
required limits? 

Solution 8: Step 1— Solve for z. 

x - x = 105 - 30 = 180 - 105 = 75 
Since a is given as 32, 
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H 25 V S xs 31 V) . 99.73 percent 7 



Figure 5-10. Probability density functions for power supply outputs. 


Step 2— From table 5-4 the area in the two tails when z ~ 2.34 
is 0.0193. Therefore, because of symmetry, 0.00965 of the 
transistors will have a 0 below 30 and 0.00965 will have a 

0 above 160. 

Step 3-Now find P ( 30 <, 0 s 180). Since 0.0193 of the 
transistors will have a 0 below 30 or above 180, then 

1 - 0.0193 must give the percentage that will lie between 30 
and 180. This is 1 - 0.0193 = 0.9807, or 98.07 percent, as 
shown in figure 5-11. If we were to buy 100 000 of these 
transistors, we would expect 98 070 of them to have a 0 
between 30 and 1 80. The remaining 1930 would not meet our 
0 requirements. 

One-Limit Problems 

In many applications engineers are interested only in one- 
sided limits, an upper or lower limit, rather than a two-sided 
upper and lower limit. In these cases they are interested in 


fl(30 S 0S18O) « 1 - 0.0193 » 0.9807 7 

/ 
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the area under one tail of the density function as shown in 
figure 5-12. Tabulated values of the area in one tail of the 
normal density function at selected values of z are given in 
table 5-5. 

Example 9: Suppose an exploding bridgewire (EBW) power 
supply is required to produce an output voltage of at least 
1500 V. At this output voltage or greater, all of the bridgewire | 
detonators will explode. If the mean output of all such supplies } 
is known to be 1575 V and the standard deviation is 46 V, j 

what is the probability that an output of 1500 V or greater ! 

will be observed? 

Solution 9: Step 1— Calculate z. > 

i 

_ Mean limit 1575 - 1500 75 j 


Step 2— Find the area in one tail of the normal curve at z from 
the mean. From table 5-5 the tail area at z = 1 .63 from the 
mean is given as 0.0516. Therefore, there is a 0.0516 prob- 
ability that an observed output will be below 1500 V. . 

Step 3— Find the probability that the output will be 1500 V j 
or greater. Since from step 2, P(x £ 1500) = 0.0516, j 

I 

P(x > 1500) = 1 - P(x £ 1500) = 1 - 0.0516 j 

= 0.9484, or 94.84 percent j 




(a) Lower limit. 

(b) Upper limit. 

Figure 3-l2.-Example of one-limit problems. 
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TABLE 5-5. —AREAS IN ONE TAIL OF NORMAL CURVE AT SELECTED VALUES OF 

(From reference 5- 1 .) 



- 

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

.0.07 

0.08 

0.09 

0 

0.5000 

0.4960 

0.4920 

0.4880 

0.4840 

0.4801 

0.4761 

0.4721 

0.4681 

0.4641 

.1 

.4602 

.4562 

.4522 

.4483 

.4443 

.4404 

.4364 

.4325 

.4286 

.4247 

.2 

.4207 

.4168 

.4129 

.4090 

.4052 

.4013 

.3974 

.3936 

.3897 

.3859 

.3 

.3821 

.3783 

.3745 

.3707 

.3669 

.3632 

.3594 

.3557 

.3520 

.3483 

.4 

.3446 

.3409 

.3372 

.3336 

.3300 

.3264 

.3228 

3192 

.3156 

.3121 

.5 

.3085 

.3050 

.3015 

.2981 

.2946 

.2912 

.2877 

.2843 

.2810 

.2776 

.6 

.2743 

.2709 

.2676 

.2643 

.2611 

.2578 

.2546 

.2514 

.2483 

.2451 

.7 

.2420 

.2389 

.2358 

.2327 

.2296 

.2266 

.2236 

.2206 

.2177 

.2148 

8 

.2119 

.2090 

.2061 

.2033 

.2005 

.1977 

.1949 

.1922 

.1894 

.1867 

.9 

.1841 

.1814 

.1788 

.1762 

.1736 

.1711 

.1685 

.1660 

.1635 

.1611 

L0 

.1587 

.1562 

.1539 

.1515 

.1492 

.1469 

.1446 

.1423 

.1401 

.1379 

LI 

.1357 

.1335 

.1314 

.1292 

.1271 

.1251 

.1230 

.1210 

.1190 

.1170 

1.2 

.1151 

.1131 

.1112 

.1093 

.1075 

. 105o 

.1038 

.1020 

.1003 

.0985 

1.3 

.0968 

.0951 

.0934 

.0918 

.0901 

.0885 

.0869 

.0853 

.0838 

.0823 

1.4 

.0808 

0793 

.0778 

.0764 

.0749 

.0735 

.0721 

.0708 

.0694 

.0681 

1.5 

.0668 

.0655 

.0643 

.0630 

.Ool? 

.0606 

.0594 

.0582 

.0571 

.0559 

L6 

.0548 

.0537 

.0526 

.0516 

.0505 

.0495 

.0485 

.0475 

.0465 

.0455 

1.7 

.0446 

.0436 

.0427 

.0418 

.0409 

.0401 

0392 

X384 

.0375 

.0367 

1.8 

.0359 

.0351 

.0344 

.0336 

.0329 

.0322 

.0314 

.0307 | 

.0301 

.0294 

1.9 

.0287 

.0281 

.0274 

.0268 

.0262 

.0256 

.0250 

.0244 

.0239 

.0233 

2.0 

.0228 

.0222 

.0217 

.0212 

.0207 

0202 

.0197 

.0192 

.0188 

.0183 

2.1 

.0179 

.0174 

.0170 

.0166 

.0162 

.0158 

.0154 

.0*50 

.0146 

.0143 

2.2 

.0139 

.0136 

.0132 

.0129 

.0125 

.0122 

.0119 

.0016 

.0113 

.01 10 

2.3 

.0107 

.0104 

.0102 

.00990 

.00964 

.00939 

00914 

.00889 

.00866 

.00842 

2.4 

.00820 

.00798 

0076 

.00755 

.00734 

.00714 

.00695 

.00676 

.00657 

.00639 

2.5 

.00621 

.00604 

.00587 

.00570 

.00554 

.00539 

.00523 

.00508 

.00494 

.00480 

2.6 

00466 

.00453 

.00440 

.00427 

.00415 

.00402 

.00391 

.00379 

.00368 

.00357 

2.7 

.00347 

.00336 

.00326 

.00317 

.00307 

.00298 

.00289 

.00280 

.007272 

.00264 

2.8 

.00256 

.00248 

.00240 

.00233 

.00226 

.00219 

.00212 

.00205 

.00199 

.00193 

2.9 

.00187 

.00181 

.00175 

.00169 

.00164 

.00159 

.00154 

.00149 

.00144 

.00139 

- 

0 

0.1 

0.2 

0.3 

0.4 

0.5 

0.6 

0.7 

0,8 

0.9 

3 

0.00135 

0.0*968 

0.0*687 

0.0*483 

0.0*337 

0.0*233 

0.0*159 

0.0*108 

0.0*723 

0.0*48 1 

4 

.0*317 

.0*207 

.0*133 

.0*854 

0*541 

0*6340 

.0*211 

.0*130 

.0*793 

.0*479 

5 

.0 h 287 

.0*170 

.0 7 996 

.0 7 579 

.0 7 333 

0 7 I9O 

.0 7 107 

.0*599 

.0*332 

.0** 1 82 

6 

.0*987 

0*530 

.0*282 

.0*149 

.0*"777 

.0*°402 

.0"*206 

.0*° 104 

.0**523 

0"260 


We can therefore expect to obtain a 1500-V output voltage 
level 94.84 percent of the time. Or to express it another way, 
94.84 percent of the supplies will produce an output above 
the minimum requirement of 1500 V. This result is shown in 
figure 5-13. Associated with the probability density function 
p(x) of the normal distribution is a cumulative probability 
distribution denoted by F(jt). As shown in the integral 
formulas of chapter 2 the relation between the two is given by 

F(x) = | p(x) dx 


So, for the norma! distribution 


F(x ) = -4= ( dx 

<jV2 t j 


or in z notation 


' W -»I 
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Figure 5-13.— Exploding bridge wire power supply output. 



A graph of F(x) is shown in figure 5- 14. Recall (hat in 
discussing cumulative functions earlier, F(x) was called 
the cumulative aicc under the density curve. Looking at 
figure 5-14, then, you can see 

(1) That F(x) = 0.5, or that 50 percent of the area under 
the normal distribution is between -oo and the "v nn Jr; 
or that there is a 50-percent probability that a variable x 
lies in the interval (— oo, x) 

(2) That 1 - F{x) = 0.5, or that 50 percent of the area 
under the normal distribution is between the mean x and 
oo ; or that there is a 50-percent probability that a variable 
jr lies in the interval (x, oo) 


(3) That the area between -la and x is 

P(-la£x£x) =F(x) - F(-\a) 

= 0.5-0.16 = 0.34 

o r that there is a 0.34 probability that a variable x will 
lie between the mean x and - la 
For more accurate work the c mulative areas for selected 
values of z have been tabulated and are shown in tables 5-6 and 
5-7. Table 5-6 shows the cumulative areas for values of z from 
- oo to 0, which is illustrated in figure 5-15. Table 5-6 shows 

(1) That at z = 0 (i.e., when the distance from the limit to 
x is 0) the cumulative area from -oo to Jc is 0.5000, 
or 50 percent 

(2) That at z — — 1 .0 the cumulative area from — oo to — la 
is 0.1587, or 15.87 percent 

(3) That at z = —2.0 the cumulative area from — ooto —2a 
is 0.02275, or 2.275 percent 

Table 5-7 shows the cumulative areas for values of z from 
0 to oo. This is illustrated in figure 5-16. 

In both tables the value of z is the same as F(jt) . It therefore 
follows 

(1) That the probability of the variable x lying between -oo 
and x is 

P(-oosx£x)=F(x)-F(-ao) 

= F(z = 0)-F(z= -oo) 

= 0.5 - 0 = 0.5, or 50 percent 

(2) That the probability of the variable or lying between —2. la 
and 3.2a is 

P{ -2.1 a £ x £ 3.2 a) = F(3.2) - F(-2.1) 

= F(z = 3.2) - F(z - -2. 1) 

= 0.9993129 - 0.01786 
= 0.9814529, or 98 percent 

Nonsymmetrical Two-Limit Problems 

The cumulative function is useful for solving nonsymmetrical 
two-limit problems, which are, in practice, the most frequently 
encountered. 

Example 10: Suppose that a time-delay relay is required to 
delay the transmission of a signal at least 90 sec but no more 
than 98 sec. If the mean "time out” of the specific type of 
relay is 95 sec and the standard deviation is 2.2 sec, what is 
the probability that the signal will be delayed within the 
specified times? 
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TABLE 5-6.— CUMULATIVE NORMAL DISTRIBUTION FROM ; = -» to 0 
(From reference 5-2. | 




0 

0.0! 

0.02 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.09 

-0 

0.5000 

0.4960 

0.4920 

0.4880 

0.4840 

0.480! 

0.4761 

0.4721 

0.4681 

0.4641 

- .1 

.4602 

.4562 

.4522 

.4483 

.4443 

.4404 

.4364 

.4325 

.4286 

.4247 

- .2 

.4207 

.4168 

.4129 

.4090 

.4052 

.4013 

.3974 

.3936 

.3897 

.3859 

- .3 

.3821 

.3783 

.3745 

.3707 

.3669 

.3632 

.3594 

.3557 

.3520 

.3483 

- .4 

.3446 

.3409 

.3372 

.3336 

.3300 

.3264 

.3228 

.3192 

3156 

.3121 

- .5 

.3085 

.3050 

.3015 

.2981 

.2946 

.2912 

.2877 

.2843 

.2810 

.2776 

- .6 

.2743 

.2709 

.2676 

.2643 

.2611 

.2578 

.2546 

.2514 

.2483 

.2451 

- .7 

.2420 

.2389 

.2358 

.2327 

.2297 

.2266 

.2236 

.2206 

.2177 

.2148 

- .8 

.2119 

.2090 

.2061 

.2033 

.2005 

.1977 

.1949 

.1922 

.1894 

.1867 

- .9 

.1841 

.1814 

.1788 

.1762 

.1736 

.1711 

.1685 

.1660 

.1635 

.1611 

-1.0 

.1587 

.1562 

.1539 

.1515 

.1492 

.1469 

.1446 

.1423 

.140! 

.1379 

-;.i 

.1357 

.1335 

.1314 

.1292 

.1271 

.1251 

.1230 

.1210 

.1190 

.1170 

-1.2 

.115! 

.1131 

.1112 

.1093 

.1075 

.1056 

.1038 

.1020 

.1003 

.09853 

-1.3 

.09680 

.09510 

.09342 

.09176 

.09012 

.08851 

.08691 

.08534 

.08379 

.08226 

-1.4 

.08076 

.07927 

.07780 

.07636 

.07493 

.07353 

.07215 

.07078 

.06944 

.06811 

-L5 

.06681 

.06552 

.06426 

.06301 

.06178 

.06057 

.05938 

.05821 

.05705 

.05592 

-1.6 

.05480 

.05370 

.05262 

.05155 

.05050 

.04947 

.04846 

.04746 

04648 

.04551 

-1.7 

.04457 

.04363 

.04272 

.04182 

.04093 

.04006 

.03920 

.03864 

.03754 

.03673 

-1.8 

.03593 

.03515 

.03438 

.03362 

.03288 

.03216 

.03144 

.03074 

.03005 

.02938 

-1.9 

.02872 

.02807 

.02743 

.02680 

.02619 

.02559 

.02500 

.02442 

.02385 

.02330 

-2.0 

.02275 

.02222 

.02169 

.02118 

.02068 

.02018 

.01970 

.01923 

.01876 

.01831 

-2.1 

,01786 

.01743 

.01700 

.01659 

.01618 

.01578 

.01539 

.01500 

.01463 

.01426 

-2.2 

.01390 

.01355 

.01321 

.01287 

.01255 

.01222 

.01191 

.01160 

.01 130 

.01101 

-2.3 

.01072 

.01044 

.01017 

.029903 

.059642 

.059387 

.(P9137 

.028894 

.028656 

.028424 

-2.4 

.(P8I98 

.027976 

.05*760 

.057549 

.057344 

.057143 

.056947 

.056756 

.026569 

.026387 

-2.5 

.026210 

.026037 

[ .055868 

.055703 

.055543 

.055386 

.055234 

.055085 

i 

.054940 

.024799 

-2.6 

.024661 

.024527 

054396 

.054269 

.054145 

.054025 

.053907 

.053793 

.053681 

.023573 

-2.7 

.023467 

.023364 

.053264 

.053167 

.053072 

.052980 

.052690 

.052803 

.052718 

.022635 

-2.8 

.022555 

.022477 

"52401 

.052327 

.052256 

.0=2186 

.052118 

.052052 

.0*1988 

.021926 

-2.9 

.021866 

.021807 

.051750 

.051695 

.051641 

.051589 

.051538 

.051489 

.0*1441 

.021395 

-3.0 

.021350 

.021306 

.051264 

.05)223 

.05)183 

.051144 

.051107 

0*1070 

.0*1035 

.021001 

-3.1 

.029676 

.0-29354 

.0*9043 

.0*8740 

,0-*8447 

.0*8164 

.0*7888 

.0*7622 

.0*7364 

.0**7114 

-3.2 

.026871 

.026637 

.0'6410 

.0*6190 

.0*5976 

.0-*5770 

.0*5571 

.0*5377 

.0*5190 

.0*5009 

-3.3 

.0-'4834 

.024665 

.0*4501 

.0*4342 

.0*4189 

.0*4041 

.0*3897 

.0*3758 

.0*3624 

.0*3495 

-3.4 

.023369 

.0’3248 

•0-*3131 

.o”c:i 

.0*2909 

.0*2803 

.0*2701 

.0*2602 

.0*2507 

.0*24 L> 

-3.5 

.022326 

.0*2241 

.0*2158 

.0*2078 

0*2001 

.0*1926 

.0*n54 

.0*1785 

.0*1718 

.0*1653 

-3.6 

.0-21591 

.0-’ 1531 

.0*1473 

.0M4P 

.0*1363 

-O’ 1311 

.0*1261 

.0*1213 

.0*1166 

.0*1121 

-3.7 

.0-’ 1078 

.02)036 

.0*9961 

.0*9574 

0*9201 

.0*8842 

.0*8496 

.0*8162 

.0*7841 

.0 4 7532 

-3.8 

.0*7235 

.0*6948 

.0*6673 

.0*6407 

.0*6152 

.0*5906 

.0*5569 

.0*5442 

.0*5223 

.0*5012 

-3.9 

.0248 10 

.0*4615 

.0*4427 

.0*4247 

.0*4074 

.0*3908 

.0*3747 

.0*3594 

.0*3446 

.0*3304 

-4.0 

.0*3167 

.0*3036 

.0*2910 

.0*2789 

.0*2673 

.0*2561 

.0*2454 

.0*2351 

.0*2252 

.0*2157 

-4.1 

.0*2066 

.0*1978 

.0*1894 

.0*1814 

.0*1737 

.0*1662 

.0*1591 

.0*1523 

.0*1458 

.0*1395 

-4.2 

.0*1335 

.0*1277 

.0*1222 

.0*1168 

.0*1118 

.0*1069 

.0*1022 

.0*9774 

.0*9345 

.0*8934 

-4,3 

.028540 

.058163 

.057801 

.0*7455 

.0*7124 

.0*6807 

.0*6503 

•0-*62l2 

.0*5934 

,0*5668 

-4,4 

.055413 

.055169 

.054935 

.0*4712 

.0*4498 

.0*4294 

.0*4098 

.0-*39! 1 

.0*3732 

.0*3561 

-4.5 

.0-*3398 

.053241 

.053092 

.0*2949 

.0*2813 

.0*2682 

.0*2558 

.0-*2439 

.0*2325 

.0*22 16 

-4.6 

.052112 

.0*2013 

.051919 

.0* ! 828 

.0*1742 

.0*1660 

.0*1581 

.0*1506 

.0*1434 

.0*1366 

-4.7 

•O’ 1301 

.051239 

.051179 

.0*1123 

.0*1069 

.0*1017 

0*9680 

.0*9211 

.0*8765 

.0*8339 

-4.8 

.0*7933 

.0*7547 

.0*7178 

.0*6827 

.0*6492 

.0*6173 

.0*5869 

.0*5580 

.0*5304 

.0*5042 

-4.9 

.0*4792 

.0*4554 

.0*4327 

.0*4111 

.0*3906 

.0*3711 

.0*35^5 

.0*3348 

.0*3179 

.0*3019 

-OP 

0 

0 

2 L 

0 

0 

0 

0 

0 

0 

0 
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TABLE 5-7. -CUMULATIVE NORMAL DISTRIBUTION FROM c = 0 to oo 
| From reference 5-2. 1 


- 

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.09 

0 

0.5000 

0.5040 

0.5080 

© 

N > 

O 

0.5160 

0.5199 

0.5239 

0.5279 

0.5319 

0.5359 

.1 

.5398 

.5438 

.5478 

.5517 

.5557 

.55% 

.5836 

.5675 

.5714 

.5753 

.2 

.5793 

.5832 

.5871 

.5910 

.5948 

.5987 

.6026 

.6064 

.6103 

.6141 

.3 

.6179 

.6217 

.6255 

.6293 

.6331 

.6368 

.6406 

.6443 

.6480 

.6517 

.4 

.6554 

.6591 

38 

.6664 

.6700 

.6736 

.6772 

.6808 

.6844 

.6879 

.5 

.6915 

.6950 

.6985 

.7019 

.7054 

.7088 

.7123 

.7157 

.7190 

.7224 

.6 

.7257 

.7291 

.7324 

.7357 

.7389 

.7422 

.7454 

.7486 

-7517 

.7549 

.7 

.7580 

.7611 

.7642 

.7673 

.7703 

.7734 

.7764 

.7794 

.7823 

.7852 

.8 

.7881 

.7910 

.7939 

.7967 

.7995 

.8023 

.805! 

.8078 

.8106 

.8133 

.9 

.8159 

.8186 

.8212 

.8238 

.8264 

.8289 

.8315 

.8340 

.8365 

.8389 

1.0 

.8413 

.8438 

.8461 

.8485 

.8508 

.8531 

.8554 

.8577 

.8599 

.8621 

U 

.8643 

.8665 

.8686 

.8708 

.8729 

.8749 

.8770 

.8790 

.8810 

.8830 

1.2 

.8849 

.8869 

.8888 

.8907 

.8925 

.8944 

.8962 

.8980 

.8997 

.90147 

1.3 

.90320 

.90490 

.90658 

.90824 

.90988 

.91149 

.91309 

.91466 

.91621 

.91774 

1.4 

.91924 

.92073 

.92220 

.92364 

.92507 

.92647 

.92785 

-92922 

93056 

.93189 

1.5 

.93319 

.93448 

.93574 

.93699 

.93822 

.93943 

.94062 

.94179 

.94295 

.94408 

1.6 

.94520 

.94630 

.94738 

.94845 

.94950 

.95053 

.95154 

.95254 

.95352 

.95449 

1.7 

.95543 

.95637 

.95728 

.95818 

.95907 

.95994 

.96080 

.96164 

.96246 

.96327 

1.8 

.96407 

.96485 

.96562 

.96638 

.96712 

.96784 

.96856 

.%926 

%995 

.97062 

1.9 

.97128 

.97193 

.97257 

.97320 

.97381 

.97441 

.97500 

.97558 

.9/515 

.97670 

2.0 

.97725 

.97778 

.97831 

.97882 

.97932 

.97982 

.98030 

.98077 

.98124 

.98169 

2.1 

.98214 

.98257 

.98300 

.98341 

.98382 

.98422 

.98461 

.98500 

.98537 

.98574 

2.2 

.98610 

.98645 

.98679 

.98713 

.98745 

.98778 

.98809 

.98840 

.98870 

.98899 

2.3 

.98928 

.98956 

.98983 

.920097 

.920358 

.920613 

.9*0863 

.9^1106 

.9^1344 

.9*1567 

2.4 

.9* 1802 

.922024 

.92240 

.922451 

.9*2656 

.9*2857 

.9*3053 

.9*3244 

.9*3431 

.9*3613 

2.5 

.923790 

.9*3963 

.9*4132 

.924297 

.924457 

.9*4614 

.9*4766 

.9*4915 

.9=5060 

.9*5201 

2.6 

.925339 

.925473 

•9 2 5604 

.925731 

.9*5855 

.925975 

.9*6093 

.9*6207 

.9*6319 

.9*6427 

2.7 

.926533 

.926636 

.926736 

.926833 

.9*6928 

.9*7020 

.927110 

.9*7197 

.9*7282 

.9*7365 

2.8 

.927445 

.927523 

.927599 

.927673 

.9*7744 

.9*7814 

.9*7882 

.9*7948 

.9*8012 

.9*8074 

2.9 

.928134 

.928193 

9 2 8250 

.928305 

.9*8359 

.9*8411 

.9*8462 

.9*8511 

.9*8559 

.9*8605 

3.0 

•9 2 8650 

.9=8694 

.928736 

.928777 

.9*8817 

.9*8856 

.928893 

.9*8930 

.9*8965 

.9*8999 

3.1 

•9 J 0324 

.920646 

.9*0957 

.9*1260 

.9*1533 

.9*1863 

.9*2112 

.9*2378 

.9*2636 

.9*2886 

3.2 

.923129 

.9*3363 

.9*3590 

.9*3810 

.9*4024 

.9*4230 

.9*4429 

.9*4623 

.9*4810 

.9*4991 

3.3 

.925 166 

.9*5355 

.9*5499 

.9*5658 

.9*5811 

.9*5959 

.9*6103 

.9*6242 

.9*6376 

.9*6505 

3.4 

.926631 

.9*6752 

.9*6869 

.9*6982 

.9*7091 

.9*7197 

.9*7299 

.9*7398 

.9*7493 

.9*7585 

3.5 

.927674 

.9*7759 

.9*7842 

.9*7922 

.9*7999 

.9*8074 

.9*8146 

.9*8215 

.9*8282 

.9*8347 

3.6 

9'8409 

.9*8469 

.9*8527 

.9*8583 

.9*8637 

.9*8689 

.9*8739 

.9*8787 

.9*8834 

.9*8879 

3.7 

.9’8922 

.9*8964 

.9*0039 

.9*0426 

.9*0799 

.9*1158 

.9*1504 

.9*1838 

.9*2159 

.9*2468 

3.8 

.9*2765 

.9*3052 

.9*3327 

.9*3593 

.9*3848 

.9*4094 

.9*4331 

.9*4558 

.9*4777 

.9*4988 

3.9 

.9*5190 

«5385 

.9*5573 

.9*5753 

.9*5926 

.9*6092 

.9*6253 

.9*6406 

*6554 

.9*6696 

4.0 

.9*6833 

.9*6964 

.9*7090 

.9*7211 

.9*7327 

.9*7439 

.9*7546 

.9*7649 

.9*7748 

.9*7843 

4.1 

.9*7934 

.9*8022 

.9*8106 

.9*8186 

.9*8263 

.9*8338 

.9*8409 

.9*8477 

.9*8542 

.9*8605 

4.2 

.9*8665 

.9*8723 

.9*8778 

.9*8832 

.9*8882 

.9*8931 

.9*8978 

.9*40226 

.9*0655 

.9*1066 

4.3 

.9*1460 

.9*1837 

.9*2199 

.9*2545 

.9*2876 

.9*7193 

.9*3497 

.9*3788 

.9*4066 

.9*^332 

4.4 

.9*4587 

.9*4831 

.9*5065 

.9*5288 

.9*5502 

.9*5706 

.9*5902 

.9*6089 

.9*6268 

.9*6439 

4.5 

.9*6602 

.9*6759 

.9*6908 

.9*7051 

.9*7187 

.9*7318 

.9*7442 

.9*7561 

.9*7675 

.9*7784 

4.6 

.9*7888 

.9*7987 

.9*8081 

.9*8172 

.9*8258 

.9*834 

.9*8419 

.9*8494 

.9*8566 

.9*8634 

4.7 

.9*8699 

.9*8761 

.9*8821 

.9*3877 

.9*8931 

.9*8983 

.9*0320 

.9*0789 

.9*1235 

.9*1661 

4.8 

.9*2067 

.9*2463 

.9*2822 

.9*3173 

.9*3508 

.9*3827 

.9*4131 

.9*4420 

.9*46% 

.9*4958 

4.9 

.9*5208 

.9*5446 

.9*5673 

.9*5889 

.9*6094 

.9*6289 

.9*6475 

.9*6652 

.9*6821 

.9*6981 

00 

1.0 

10 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 | 
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Figure 5-15. — Cumulative areas for values of z from -od | 0 0. 
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Figure 5- 1 6. —Cumulative areas for values of c from 0 to < 
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Figure 5-17.— Signal delay time. 


Solution 10: Step I— Find Fl(98 sec). Since the mean is given 
as 95 sec and the standard deviation as 2.2 sec. 


_ Limit - Mean 98 -95 3 

1 — : 


From table 5-7 

F(98 sec) = F(z) = F(1.36) = 0.91309 


Step 2— Find F(90 sec). Since the mean is 95 sec and the 
standard deviation is 2.2 sec, 


90 - 95 -5 

• ® — - -2.27 


2.2 2.2 


From table 5-6 

F(90 sec) = F(z) = F(- 2.27) = 0.01 160 
Step 3-Find F( 90 sx< 98). From steps I and 2 
F(90 < .r < 98) = F(98) - F( 90) = 0.91309 - 0.01 160 

— 0.90149. or 90 percent 

There exists, therefore, a 90-percent probability that the signal 
will be delayed no less than 90 sec and no more than 98 sec. 
This is shown in figure 5-17. 


Application of Normal Distribution to Test Analyses and 
Reliability Predictions 


This section gives two examples of how the normal 
distribution techniques may be applied to the analysis of test 
data of certain devices and how the results of the analysis may 
be used to estimate or predict the outcome of actual tests (ref. 
5-5). Many similar examples are given in the next chapter 
Example 11: For this two-limit problem, assume that a door 
hinge has a pin pull-force requirement of 12 ±4.64 lb. Assume 
further that we have received 116 door hinges and have 
actually measured the pin pull-force required for 16 of them 
art of an acceptance test. The results of the test are as 
shovn in table 5-8 and in histogram form in figure 5-18. We 
now want to apply normal distribution theory and then estimate 
what percentage of the remaining 100 door hinges will meet 
the pin pull-force requirement. 

Solution 11: Step 1— Solve for the mean of the test data x. 
We have already seen that 


x = 


n 

E* 

i=i 


where 

x > value of / ,h measurement 
n total number of measurements 


TABLE 5-8.— RESULTS OF DOOR HINGE 
ACCEPTANCE TEST 


Pull-force 

required, 

lb 

Number of 
occurrences 1 

8 

1 

10 

3 

12 

7 

14 

4 

16 

I 

Total 

16 


jS1 


! i 


' O 


. < 



Lower 
acceptance 
fimit (-2.32a) 


' 1 percent 
will be 
■ defectivo 


r Area under density 
i function between 
acceptance limits, 
r 98 percent 


Upper 

■ acceptance 
I imit (+2.32 o) 


r 1 percent 
wilbe 
defective 
here 


2 4 6 8 10 12 14 16 16 20 22 

Pin pulMoroe. lb 


Solve for E fa - xf: 

(=1 ' /= | v ' 


= (8 - 12) 2 + 3(10 - 12) 2 + 7(12 - 12) 2 
+ 4(14- I2) 2 + (I6- I2) 2 
- ( 4) 2 + 3(-2) 2 + 7(0) 2 + 4(2) 2 + (4) 2 
= 16+12+0+16+16 = 60 


-4o -3o -2c -la x 1o 2o 3o 4o 
Figure 5-18.— Door hinge iesl retails. 




Then solve for 


Let v = pound forces so that 

x, = 8 
x 2 = 10 
x, = 10 
x 4 = 10 
^5 = 12 
x 6 = 12 
x 7 = 12 
Xh = 12 


X9 =12 
X|o - 12 

-X|. = 12 
X| 2 = 14 
x l3 = 14 
X|4 = 14 
X|5 = 14 

X| 6 = 16 


and let n - 16 (number of occurrences). The mean x is therefore 


i _ _!z!_ m 8 + 3(10) + 7(12) +4(14) + 16 
» 16 

= 12 lb (rounded to two places) 

Step 2 Solve for the standard deviation o . We have also seen that 


p \* i -*) 2 


x observed mean 
x value of / ,h measurement 
n total number of measurements 




1 60 60 ^ 

n “ I 16-1 15 4 


Finally solve for a : 


£ if, - iif 

i=l ' 

; = V4 = 2 lb 


Step 3 With a mean of x = 12 lb and a standard deviation 
of a = 2 lb. figure 5-18 shows 

(1) That the lower pull-force limit of 7.36 lb is z = 
(7.36 — l2)/2 = —2.32 standard deviations from the mean 

(2) That the imner limit of 16.64 lb is z = (56.64 - !2)/2 
= 2.32 standard deviations from the mean 

Consequently, the percentage of door hinges that should fall 
within the 12 ± 4.64-lb tolerance is given by 

P( -2.32(7 <; x s ;_2.32 .ct) = F(2,32) - F(- 2.32) 

= 0.98983 - 0.01017 

(from tables 5-6 and 5-7) 

= 0.97966, or 98 percent 

This says that 98 percent of the door hinges should fall within 
the 12 ± 4.64-lb tolerance and that 2 percent should be outside 



of the required tolerance. However, none of the 16 samples 
were outside the tolerance. So where are the 2 percent that 
the analysis says are defective? The answer is that the 2 percent 
of defective door hinges are in the 100 not tested. 

We can make this statement by assuming that if we had tested 
all 100 door hinges, we would have expected to observe the 
same mean, x = 12 lb. and standard deviation, a = 2 lb. as 
we did with the 16 samples. (This assumption is subject to 
confidence limits discussed in chapter 6.) If we accept this 
assumption, we would expect to find two of the 100 door 
hinges defective: one would have a pull-force less than 7.36 lb 
(the lower limit); and one, a pull-force greater than 16.64 lb 
(the upper limit). This is also shown in figure 5-18. 

However, considering the 16 door hinges to be actually 
representative of all such door hinges, we could predict that 
only 98 percent of such door hinges produced would meet the 
acceptance criteria of a 12 ± 4.64-lb pin pull-force. 

Example 12: In this one-limit problem, 10 power supplies 
are selected out of a lot of 110 and tested at increasing 
temperatures until all exceed a maximum permissible output 
of 31 V. The failure temperatures in degrees centrigrade of 
the 10 supplies are observed to be 


X, -= 57 

*6 =60 

-r 2 = 65 

£ 

ll 

.t, = 53 

* 8 =82 

2 

ii 

x 9 = 71 

Jj = 66 

Xjo = 69 


Find the probability that the remaining 100 supplies will have 
an output greater than 31 V at 50 °C and below. 

Solution 12: Step 1 — Solve for the mean Jr. 

10 

E.v, 

? _ 57+65+53+62+66+60+75+82+71 +69 

10 10 



“C 


Step 2— Solve for the standard deviation a. First, 


Area above 50 c C is probability 
that output will not be greater than 
31 V at 50 °C and below: P« 0.96712 

/ 

Area below 50 °C is / 



Figure 5- 19. —Failure distribution of power supplies. 


Then 


10 


n- I 


— 8.7 deg C (rounded to two places) 


Step 3— Solve for z = (Limit - Mean)/a. With an observed 
mean of x - 66 and a standard deviation of a = 8.7, the 50 °C 
limit is z = (50 - 66J/8.7 = - 16/8.7 = - ] .84 observation 
locations in standard deviations from the mean. 

Step 4 Look at table 5-6 and find the cumulative area from 
-oo to o = - 1 .84. This is given as 0.03288. Therefore, there 
is a 3.288-percent probability that the remaining 100 supplies 
will have an output greater than 31 V at 50 # C and below. 
This is shown in figure 5-19. 



Effects of Tolerance on a Product 


10 

L(xt - 66 j 2 = (57 - 66) 2 + (65 + 66) 2 + (53 - 66) 2 
+ (62 - 66) 2 + (66 - 66) 2 + (60 - 66) 2 
+ (75 - 66)“ + (82 - 66) 2 + (71 - 66) 2 + (69 - 66) 2 
= 81 + 1 + 169+16 + 0 + 36 + 81 

+ 256 + 25 + 9 

= 674 


(1) What car tolerances do to affect the reliability of a product ? 

(2) How can tolerances be analyzed? 

(3) What methods are available? 

(4) What will affect the term P, in the product reliability 

model? ' 

These questions are important to ask because tolerances must 
be expected in all manufacturing processes. 

Electrical circuits are often affected by part tolerances (i.c., 
circuit gains can shift up or down, and transfer function poles 
zeros can shift into the right-hand .r-plane, causing 
oscillations). Mechanical components may not fit together or 
may be so loose that excessive vibration causes trouble (refs. 
5-6 to 5-8). 
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Notes on Tolerance Accumulation: A How-To- Do-It Guide 

General.— The notation used in calculating tolerance is 
T tolerance 

o v standard deviation 

V dependent variable subject to tolerance 

accumulation 

x independent, measurable parameter 

l.2.3.n subscript notation for parameters 
/ generalized subscript (i.e., / = 1 , 2,3 n for x t ) 

Tolerance is usually ±3<j. When in doubt, find out. Note that 
when T is expressed in percent, always convert to engineering 


units before proceeding. The mean or average is V = f(x h 
■*>*3- • ••*«)• The coefficient of variation is C, = (o/V) 
x 100 = percent. 

Worst-case method — The worst-case method is as follows: 


V=J{(x i + T x )Ax, + T 2 ),(i 3 + h) (T„ + T„)] 

-''=m,-T { )Ah ~ T 2 )Axj - Ty), . . (jf„ - 7),)) 


Actually, 

±^=/l(ii ± ± t 2 )Ax } ± r 3 ) (x„ ± r,,)] 


where the plus or minus sign is selected for maximum Kand 
then selected to give minimum V. If these ± V worst-case limits 
are acceptable, go no further. If not, try the root -sum-square 
method. 

Root-sum-square method.— The root-sum-square method 
is valid only if the/fx’s) are algebraically additive (i.e., when 
I' is a linear function of the *’s): 


± V = V ± 3o 


where 


Ov — 0~\ + 02 + (Jy + ... -f 


and 


„ _T, 

""I 


if T, = ±3c 


Stated another way 


±y = v± 




1/2 


If these ± V root-sum-square limits are acceptable, go no 
further. If they are not acceptable or the f(x's) involve products 
or quotients, try the perturbation or partial derivative methods. 








±V=V±3o v 


where 


= ( p *. - v f + -vf+...+ (P Atn - vf 

and where 

=/[ (xi ± 0 \), (x 2 ± o 2 ) y (*3 ± a 3 ) (Jr M ± a M )J 


The ± V limits are valid if C, = (a v fV) x 100 < 10 percent. 

Partial derivative method .— The partial derivative method 
is as follows: 


±V=V±3o v 


where 


2 dv\ , fdv \ 2 , / 3 K \ 2 , 

\dxj \dx 2 J - \dxj “ 


The ± V limits are valid if C,. = (o/V) x 100 < 10 percent. 

Thus, four methods are available for estimating the effects 
of tolerance on a product. The worst-case method can be used 
on any problem. In those cases where the ± V worst-case limits 
are not acceptable, other methods can be tried. The root-sum- 
square method is usually valid if the functions are algebra- 
ically additive. The perturbation or partial derivative methods 
are valid only if the coefficient of variation is less than or equal 
to 10 percent. 


Estimating Effects of Tolerance 


The following examples illustrate how these tolerance 
equations can be used. Consider a stacked tolerance problem 
where the dependent variable is a linear function— three variables 
added to give V. 


y — .V| + x 2 + 

T= 3a 


where 


X| = 1 ± 0. 1 mil 
x 2 — 2 ± 0. 1 mil 
Xy = 3 ± 0 . 1 mil 

Now, find V and the expected range of V. 


Yv. T 



Perturbation method. —The perturbation method is as follows: 


i 
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V — I + 2 + 3 = 6 mils 

Using the worst-case method, with positive tolerance 
V+ = (1 + 0.1) + (2 +0.1) + (3 + 0.1) = 6.3+ 

and with negative tolerance 

V- = (1 - 0. 1) + (2 - 0. 1) + (3 - 0. 1) = 5.7_ 
or 

y± = 6 ± 0.3 mil 

In the worst-case method the tolerance on V (i.e., 0.3 mil) 
is worse than the 3a,. tolerance. Tolerance can and often does 
cause fit problems and circuit problems. Therefore, in some 
cases we need to know what tolerance is acceptable. 

Using the root-sum-square method, 

V = 6 mils 
and 

0.1 _ 

a \ - — = 0.033 = a 2 = <r 3 

°v = (ff| + o\ + a 3 2 ) ,/2 = (3a]) l/2 

[3(0.033) 2 ] 1/2 = 0.0572 
3a, = 0.172 


Using the worst-case method, 

v ± =(10 ± 1) x (5 ± 0.5) x (2 ± 0.1) = 11 x 5.5 x 2.1 

or 9 x 4.5 x 1.9= 127 or 77 

The root-sum-square method cannot be used because these 
variables are not algebraically additive. 

Using the perturbation method, 

V = V ± 3a, 

where 



so that 

= 6 ± 0.172 mils 

In the root-sum-square method, the T value of 0.172 is the 
3a tolerance on V. 

As a second example, consider a volume problem that has 
three variables in multiplication. Find Kand the expected range 
of V. 

' i y = LWH - 10 ft X 5 ft X 2 ft = 100 ft 2 

4 

_i convert percent tolerances to engineering units: 


a ' ~ [[(10 + 0.33)(5)(2) - 10 '] 2 + [(5 + 0.17)(10)(2) 

- 100J 2 + [ 2 + 0.03)(1 0)(5) - 100] 2 ] 1 ' 2 

= [(100.3 - 100) 2 + (103.4 - 100) 2 + (101.5 - 100) 2 j 
= (10.89 + 1 1 .56 + 2.25) 1/2 = V25 = 5 
y= V ± 3a, = 100 ± 15 ft 3 
Checking the validity gives 




L= 10ft± 
H^=5 ft± 
H = 2 ft ± 


• t 


V'l 


I 






10 percent = 10 ft ± 10 ft x 0.1 = 10 ft ± 1 ft 
10 percent = 5 ft ± 5 ft x 0. 1 = 5 ft ± 0.5 ft 
5 percent = 2 ft ± 2 ft x 0.05 = 2 ft ± 0. 1 ft 
T= ±3 a 







c '' - T ~ I5o x iq: = 5 percent 

which is less than 10 percent. This solution is a better estimate 
of the effects of tolerance on volume. Note too that various 
values can now be estimated for different types of problems 
regarding this volume because it has been represented as a 
normal distribution function. 


»v> 




Using the partial derivative method, again 
V ± -V ± 3a, 

where 



dV dV dV 

v = LWH ’ aZ = WH ' dw~ LH ' m ~ LW 

a L = 0.33 ft, a w = 0. 17 ft, a H = 0.03 ft 
ay = |(WH)i a\ + ( LH)w a& + ( LW)~ H (T#j 
= [(5 x2) 2 (0.33) 2 + (10x2) 2 (0.17) 2 
+ (10 X 5) 2 (0.03) 2 ] 

= (l0.9 + 1 1.6 + 2.25)' /2 = n/25 = 5 
V= 100 ± 15 ft 3 

This method is more work and gives the same results as the 
perturbation method. Because the C v - 5 percent, which is 
less than 10 percent, the method would be suitable to use. 


Concluding Remarks 

Now that you have completed chapter 5 you should have 
a clear understanding of the following concepts: 

(1) A probability density function pU) for a random vari- 
able describes the probability that the variable will take on 
a certain range of values. 

(2) The area under the density function is equal to unity, 
which means that the probability is 1 that the variable will be 
within the interval described by the density function. For 
example, the normal distribution describes the interval from 
—oo to oo. 

(3) Associated with each probability density function is a 
cumulative probability distribution F(x ) that represents the 
cumulative sum of the areas under the density function. 

(4) The normal distribution (also called the bell curve, the 
Gaussian distribution, and the normal curve of error) is a 
probability density function. Using the normal distribution, 
you should be able to solve the following types of problems: 



(a) Symmetrical two-limit problem:,, which are concerned 
with the probability of a variable taking on values 
within equal distances from both sides of the mean 



(b) Nonsymmetrical two-limit problems, which are similar 
to (a) but within unequal distances from both sides of 
the mean of the density function 




(c) One-limit problems, which are concerned with the 
probability of a variable taking on values above or 
below some limit represented by some distance from 
the mean of the density function 

(5) You should be able to take data measurements of a 
certain device and calculate the mean of the data given by 



n im | 
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and Che standard ueviation of the data given by 



i=l n ~ 1 


(8) The perturbation or partial derivative methods are only 
valid if the coefficient of variation is 10 percent or less. 


S. 


and 
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Using the data mean and standard deviation, you should then 
be able to estimate the probability of failures occurring when 
more of the same devices are tested or operated. 

(6) The worst-case method can be used on any problem: 

(a) Limits will be defined. 

(b) No estimates can be made from the population 
distribution. 

(7) The root-sum-square method only applies to algebraic 
variables that are additive. 
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Reliability Training 1 

1. A unit is required to operate at 100 °F. If tests show the mean strength of the unit is 123 °F, and the 
standard deviation is 9 deg F, what is the probability that the unit will operate successfully that is 
p (x > 100 °F)? 

A. 0.5234 B. 0.2523 C. 0.9946 D. 0.9995 

2. A pressure vessel (including a factor of safety) has an upper operating limit of 8000 psi. Burst tests 
show a mean strength of 9850 psi and a standard deviation of 440 psi. What is the probability of pressure 
vessel failure; that is, P(x < 8000 psi)? 

A. 0.0 4 267 B. 0.0 4 133 C. 0.0 4 3I7 

3. A memory drum is required to reach sink speed and stabilize in 15.5 sec at 125 °F. Five drums are 
tested with these stabilizing time results; 13.2 sec, 12.3 sec, 14.8 sec, 10.3 sec, and 12.9 sec. 

a. What is the mean stabilizing time? 

A. 13.1 B. 10.7 C. 12.7 

b. What is the standard deviation? 

A. 1.63 B. 1.45 C. 1.32 

c. What is the estimated percentage of drums out of specification; that is, P(x > 15.5 sec)? 

A. 6.7 B. 8.5 C. 4.3 

4. A pyrotechnic gyro has an uncaging time requirement of 142 ± 20 msec. Six gyros were tested resulting 
in these uncaging times: 123, 153, 140, 129, 132, and 146 msec. 

a. What is the mean uncaging time? 

A. 133.2 msec B. 135.2 msec C. 137.2 msec 

b. What is the standard deviation? 

A. 10.2 B. 11,2 C. 11.9 

c. That is the estimated percentage of gyros within specification; that is, P( 122 < jr < 162 msec)? 

A. 89.8 B. 96.8 C. 82.6 

5. A hydraulic pressure line was designed to the following stresses; 

(a) Maximum operating pressure (actual), 1500 psi 

(b) Design pressure (10 percent safety factor), 1650 psi 

Tests of the pressure line indicated a mean failure pressure of 1725 psi and a standard deviation of 45 psi. 

a. What is the reliability of the line when the design pressure limits are considered? 

A. 0.10 B. 0.90 C. 0.98 

b. What is the reliability of the line when the maximum operating pressure is considered? 

A. 0.99 B. 0.90 C. 0.80 


'Answers are given at the end of this manual, 


6. A communications network requires a 1 300-msec watchdog delay after initiation. A sample of 10 delays 
wer„ tested from a rack of 100 delays. The time delays of the circuits are as shown: 


1 Circuit 
number 

Delay, 

msec 

1 

1250 

2 

1400 

3 

1700 

4 

1435 

5 

1100 

6 

1565 

7 

1485 

8 

1385 

9 

1350 

10 

1400 


a. What is the average (mean) delay time? 

A. 1386 msec B. 1400 msec C. 1407 msec 

b. What is the standard deviation? 

A. 52.7 B. 87.1 C. 163.4 

C or^eateTd^y)? 8 SamP,e ’ What PerC£ntage ° f ,he 100 circuits wil1 meet specifications (1300 msec 
A. 75 B. 80 C. 90 


7. A circuit contains four elements in series. Their equivalent resistance values are 


Element 

Nominal 

resistance, 

R, 

ohm 

Tolerance, 11 

T, 

percent 

A 

100 

±10 

B 

20 

±1 

C 

10 

±5 

D 

10 

±5 1 


“Where ±T= ±3 a. 


a. What is the nominal or mean total resistance R T 1 
A. 120 Q B. 140 0 C. 160 0 

b. What are the worst-case R values? 


A +l3L6 0 
A ’ -118.7° 


B + * 76.3 0 
B ' -146.2° 


c. +,5, - 2 a 

-128.8“ 


c. Using the root-sum-square method, what is the probability that R T & 135 0? 
A. 0.905 B. 0.962 C. 0.933 


d. Using the perturbation method, what is the probability that R T a 135 0? 
A. 0.905 B. 0.962 C. 0.933 
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a. Whal is the nominal or mean power output P'i 
A. 25 VV B. 20 W C. 30 W 

b. What are the worst-case P values? 


A. 


+26.6 

-18.2 


W 


B. 


+35.2 

- 22.6 


W C. 


+30.3 

-20.3 


W 


c. Using the perturbation method, what is the probability that (23.5 < p < 26.5)? 

A. 0.94 B. 0.80 C. 0.86 

d. What is the C, (in percent) for the perturbation method used in question 8c? 

A. 12% B. 8% C. 46% 

e. Is the rou, ,m-square method valid for solving the probability problem 8c? 

A. Yes B. No 

f. Using the partial derivative method, what is the probability that (23.5 <P< 26.5)? 

A. 0.942 B. 0.803 


± 10 percent. 


C. 0.857 


Chapter 6 

Testing for Reliability 

In chapters 3 and 4 we discussed the methods used to predict 
the ptobability that random catastrophic part failures would 
occur in given products and systems. These analytical tech- 
niques are well established (ref. 6-1). Yet, we should keep 
in mind that they are practical only when adequate exper- 
imental data are available in the form of part failure rates. In 
other words, their validity is predicated on large amounts of 
empirical information. 

Such is not the case when we undertake similar analyses 
to determine the influence of tolerance and wearout failures 
on the reliability of a product. An understanding of these 
failure modes depends on experimental data in the form of 
probability density functions such as those discussed in 
chapter 5. In general, such data are unavailable on items at 
the part or system level; this kind of information must be 
developed empirically through reliability test methods. 

Chapter 6 reviews and expands on the terms used in the 
reliability expression given in chapter 2 and then shows how 
the terms can be demonstrated or assessed throi gh the appli- 
cation of attribute test, test-to-failure. and life test methods 
(ref. 6-2). 


Demonstrating Reliability 

Recall from chapter 2 that one way to define product 
reliability is as the probability that one or more failure modes 
will not be manifested (ref. 6-3). This can be written as 

* = WAWM*.,) 

where 

P,- probability that catastrophic part failures will not occur 
P, probability that out-of-tolerance failures will not occur 
P w probability that wearout failures will not occur 

Ky probability that quality test methods and acceptance 
criteria will not degrade inherent reliability 

K,„ probability that manufacturing processes, fabrication, 
and assembly techniques will not degrade inherent 
reliability 


K r probability that reliability engineering activities will not 
degrade inhei reliability 

K t probability that logistics activities will not degrade 
inherent reliability 

K„ probability that user or customer will not degrade 
inherent reliability 

The term P,P,P W denotes inherent reliability R,\ (K^KJCfK,,) 
are factors that affect the probability of the three modes of 
failure occurring during hardware manufacture and use rather 
than from unreliable hardware design. 

First, we illustrate how the empirical value of these terms 
affects product reliability. Then, we discuss the particular test 
methods used to develop these values. Assume that a device 
was designed with a reliability requirement of 0.996. This 
means that only four out of 1000 such devices can fail. The 
device contains 1000 parts, it has a function to perform within 
a tolerance ot X ± 2 percent, and it must operate for a mission 
cycle of 1000 hours at 50 °C. 

P c Illustrated 

If we know the number and types of parts in the device plus 
the applied stresses and part failure rates used in the exponen- 
tial distrit jtion, e - ' 15 * 1 , we can estimate the probability that 
no catastrophic part failure will occur during the mission cycle. 
Assuming, for example, that our estimate is P, = 0.999 (Le- 
one device in 1000 will incur a catastrophic part failure during 
the mission cycle), the product reliability of the device becomes 


R = P,P ,P „ ( K factors) = e n ~ h P,PJK factors) 

= 0.999/»A (AT factors) 

P, Illustrated 

Suppose we now test one of the devices at 50 *C. If the 
functional output is greater than the specified tolerance of 
A( ± 2 percent, the reliability of that particular device is zero, 
It is zero because P, is zero (i.e.. R = (0.999)(0 )/>, (K factors) 
= 0). We can say. however, that the device will continue to 
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operate in an out-of-tolerance condition with a probability of 
no catastrophic failures equal to 0.999 just as we predicted. 

To understand this better, recall that part failure rates reflect 
only the electrical, mechanical, and environmental stresses 
applied to the individual parts. For this reason a prediction 
on the basis of such data vill neglect to indicate (1) that 
the parts have been connected to obtain a specified function, 
(2) that a tolerance analysis of the function has been per- 
formed, or (3) that the parts are packaged correctly. In other 
words, P c represents only how well the individual parts will 
operate, not how well the combined parts will perform. 

If nine more of the devices are tested at 50 °C with all the 
output functions remaining within the X ± 2 percent tolerance, 
P t becomes 9/10 = 0.9 and the reliability of the device 
R - (0.999)(0.9)P m , (K factors). Because the reliability 
requirement of the device is 0.996, it should be clear that P t 
must be greater than 0.996. Let us assume then that 1000 
devices are tested at 50 °C with only one tolerance failure, 
which produces an observed P \ -- 999/1000 = 0.999. The 
reliability of the device is now 

R = (0 . 999)(0 . 999)P W . ( K factors) = 0.998 P W (K factors) 

Note that, because operating time is accumulated during original 
functional testing, it is possible for random catastrophic part 
failures to occur. Remember, however, that this type of failure 
is represented by P c and not P r 

P w Illustrated 

Now let us take another operating device and see whether 
wearout failures will occur within the 1000-hour mission cycle. 
If, as run time is accumulated, a faulty function output 
or catastrophic failure is caused by a wear mechanism , the 
reliability of the device again becomes zero. It is zero because 
P w is zero as shown in the equation 

R = (0.999)(0.999)(0)(AT factors) = 0 

Note the emphasis on the words “wear mechanism. M Because 
it is possible to experience random catastrophic part failures 
and even out-of-^erance conditions during a test for wearout, 
it is absolutely necessary to perform physics-of-failure analyses. 
This is essential in ascertaining whether the failures are 
caused by true physical wear before including them in the P w 
assessment. 

So far, the first two terms, P c and P n combine to yield a 
probability of (0.999)(0.999) = 0.998. As a result, the 
remaining terms, P, v ( K factors), must be no less than 0.998 
if the 0.9% device requirement is to be satisfied. Therefore, 
we assume that we have demonstrated a P w of 0.999, which 
reduces the device reliability to 

* = W h <* factors) * (0.999)(0.999)(0.999)(tf factors) 

» 0.997(tf factors) 


K Factors Illustrated 

Since testing obviously must be conducted on real hardware, 
the K factors as well as the P terms of reliability are present 
in every test sample. Establishing values for the K factors 
requires that all failures observed during a test be subjected 
to physics-of-failure analyses by which specific failure mech- 
anisms are identified. Actually, the action taken to pievent 
the recurrence of an observed failure mechanism determines 
the factor that caused the failure. A failure that can be f 
prevented by additional screening tests as part of the quality 
acceptance criteria is charged to the K q factor; one that 
requires additional control over some manufacturing process ) 
is charged to the K m factor, etc. Failures that require changes 
in documentation, design, and tolerance would be charged to 
the P c , P Jf or P w terms as applicable. 

The least important aspect of testing is the ability to charge 
an organization or function with responsibility for a failure. 
More important is the need to prevent observed failures from 
recurring. This requires that corrective action be made a 
recognized part of each reliability test program. j 

Getting back to the illustration, we assume that one failure j 
out of 1000 devices was caused by one of the K factors even j 
though it could have been observed during a P c , P tt or P w j 
failure evaluation. This reduces the reliability of the device to j 

f 

R-P ( P,P„UC factors) =(0.999)(0.999)(0.999)(0.999) =0.9% ' 

1 

which indicates that the device met its requirement. 

Test Objectives and Methods ' 

The purpose of the preceding illustration was to provide a 
better understanding of (1) how the P terms and the K factors 
relate to physical hardware and (2) the techniques for demon- 
strating the terms through testing. Table 6-1 shows the 
suggested test methods. We say “suggested” because any of 
the test methods can be used if certain conditions are met 
(ref. 6-4). These conditions are pointed out as each method 
is discussed. Table 6-1 indicates the most efficient methods 
by assigning priority numbers from 1 to 3 (with 1 being the 
most efficient and 3 the least). 


TABLE 6- 1 .-TEST METHOD PRIORITIES 
FOR DEMONSTRATING RELIABILITY 


Reliability 

term 

Suggested test method 

Attribute 

tests 

Tests to 
failure 

Life 

tests 

r. 

2 

3 

l 

p, 

3 

1 

2 

p»- 

3 

2 

I 

K factors 

3 

l 

2 
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Test Objectives 

c rom our discussions thus far it can be inferred that 1000 
test samples are required to demonstrate a reliability require- 
ment of 0.999. Because of cost and time considerations this 
is obviously an impractical approach. Furthermore, the total 
production of a product often may not even approach 1000 
items. Because we usually cannot test the total production of 
a product (called product population), we must demonstrate 
reliability on a few samples. Thus, the main objective of a 
reliability test is to test an available device in such a way that 
the data will allow a statistical conclusion to be reached about 
the reliability of similar devices that will not or cannot be 
tested. In other words, the main objective of a reliability test 
is not only to evaluate the s*>ecific items tested, but also to 
provide a sound basis for predicting the reliability of similar 
items that will not tr tested and that often have not yet been 
manufactured. 

In chapter 2 we explained that to know now reliable a 
product is you must know how many ways it can fail and the 
types and magnitudes of the stresses that produce such failures. 
This premise leads to a secondary objective of a reliability test, 
which is to produce failures in the product whereby the types 
and magnitudes of the stresses that r.ause such failures are 
identified. It follows then that reliability tests that result in no 
failures provide some measure of eliability but little infor- 
mation about the population failure n echanisms of like devices. 
(There are exceptions, of course, is pointed out later.) 

In the subsequent sections of his chapter, we discuss 
attribute test, test-to-failure, and life test methods, explain how 
well these methods meet the test objectives just described, 
show how tH test results can be statistically analyzed, and 
introduce the subject and use of confidence limits. A good 
discussion of reliability testing for demonstration purposes is 
given in MIL-STD-785B (ref. 6-1). 

Attribute Test Methods 

Qualification, preflight certification, and design verification 
tests fall in the category of attribute tests (refs, 6-4 and 6-5). 
They are usually of the go'no-go type used to demonstrate that 
a device is good or bad without showing how good or how bad 
it may be. In a typical test two samples are subject*" 1 to a 
selected level of environmental stress, usually the maximum 
anticipated operational limit. If both samples pass, the device 
is considered qualified, preflight certified, or verified for use 
in the particular environment involved (refs. 6-6 and 6-7). 
Occasionally, such tests are called tests to success because the 
true objective is to have the device pass the test. 

This can be illustrated by the example of two power supplies, 
each with an output requirement of 12 ± 0.24 V at a maximum 
temperature of 125 °F. If we test these items at 125 °F, we 
might observe an output of 12.230 V for one and 12.215 V 
for the other. Since the output of each supply falls within the 
required tolerance, we would call both qualified, or preflight 
certified, as the case may be. This might seem to be a 


declaration that all similar supplies, including any not yet built, 
would also pass the test and be within the tolerance limit of 
125 °F. But no such statement would be valid from the results 
of so simple a test. The only reasonable conclusion we can 
reach from testing two samples to success is that these items 
alone are qualified. 

Confidence levels.— Mr. Igor Bazovsky in his book entitled 
“Reliability Theory and Practice" (ref. 6-2) helps us to 
understam what the term “confidence" means in the business 
of testing: 

We know that statistical estimates are more likely to be 
close to the true value as the sample size i.icreases. Thus, 
there is a close correlation between the accuracy of an 
estimate and the size of the sample from which it was 
obtained. Only an infinitely large sample size could give 
us a 100 percent confidence or certainty that a me* „ured 
statistical parameter cr ; !.cides with the true value. In this 
context, confidence is a mathematical probability relating 
the mutual positions of the true value of a parameter and 
its estimate. 

When the estimate of a parameter is obtained from a 
reasonably sized sample, we may logically assume that the 
true value of that parameter will be somewhere in the 
neighborhood of the estimate, to the right or to the left. 
Therefore, it would be more meaningful lo express statis- 
tical estimates in terms of a range or interval with an 
associated probability or confidence that the true value lies 
within such interval than to express them as point estimates. 

This is exactly what we are doing when we assign con- 
fidence I* Tiits to point estimates obtained fr m statistical 
measurements. 

To illustrate further the limitations of attribute test methods, 
wc apply ‘ ; stics to the test results. Figure A-4(a) in appen- 
dix A shows on the ordinate the number of events (successes) 
necessary to demonstrate a reliability value (abscissa) for 
various confidence levels (family of curves) when no failures 
are observed. Figures A-4(b) to (0 provide the same infor- 
mation when one to five failures are observed. 

From die results of two devices tested with no failures, figure 
A-4(a) shows that we can state with 50-percent confidence 
that the population reliability of such devices is no less than 
71 percent. Fifty-percent confidence means that there is a 
50-percent chance that we are wrong and that the reliability 
of similar untested devices will actually be less than 71 percent. 
Similarly, we can also state from the same figure that we are 
60 percent confluent that the reliability of all such devices is 
63 percent. Hut either way the probability of success is less 
than encouraging. 

To gain a better understanding of figure A-4 and the theory 
behind it, let us stop for a moment and see how confidence 
levels are calculated. Recall from chapter 2 that the com- 
bination of events that might result from a test of two devices 
was given by 

R 2 + IRQ + Q ? = 1 
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where 

® probability that both devices will pass 

2RQ probability that one device will pass and one will fail 

Q probability that both devices will fail 

In the power supply example we observed the first event Rr 
because both supplies passed the test. If we assume a 50-percent 
probability that both will pass, we can set R 2 = 0.50 and 
solve for the reliability of the device as follows: 

R 2 = 0.50 


R = Vo. 50 = 0.71 


We then can say with 50-percent confidence that the population 
reliability of the device is no less than 0.71. By assuming a 
50-percent chance, we are willing to accept a 50-percent risk 
of being wrong, hence the term "50 percent confident." If 
we want only to take a 40-percent risk of being wrong, we 
can again solve for R from 


R 2 = 0.40 


R = Vo .40 = 0.63 

In this case, we can be 60 percent confident that the population 
reliability of the devices is no less than 0.63. 

Selection of the confidence level is a customer's or 
engineer's choice and depends on the amount of risk they are 
willing to take on being wrong about the reliability of the 
device. The customer usually specifies the risk he or she is 
willing to take in conjunction with the system reliability 
requirement. As higher confidence levels (lower risk) are 
chosen, the lower the reliability estimate will be. For example, 
it we want to make a 90-percent confidence (10-percent risk) 
statement based or. the results of the test to success of two 
devices, we simply solve 

/f 2 = (I - Confidence level) = 1 - 0.90 = 0.10 
so that 

R = VO. 10 = 0.316 


Table 6-2 illustrates how the reliability lower bound changes 
with various confidence levels. The curves in figure A-4 are 
developed in a similar manner. In figure A-4(b), which is used 
when one failure is observed, for 10 samples tested with one 
observed failure the statistically predicted or demonstrated 
reliability at 90-percent confidence is 0.66. This answer is 
found by solving 

/? 10 + IO/?’e = I - 0.90 

R = 0.663 

which agrees with the figure to two places. 
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TABLE 6-2.-REUABIUTY AND CONFIDENCE 
LEVEL FOR TWO-SAMPLE ATTRIBUTE 
TEST WITH NO FAILURES 


Confidence 

level, 

percent 

Reliability, 

R 

Risk. 

percent 

in 

0.95 

90 

50 

.71 

50 

60 

.63 

40 

70 

.55 

30 

80 

.45 

20 

90 

.32 

10 

99 

.10 

1 


Application.— The discussion thus far has underscored the 
shortcomings of attribute tests when sample sizes are small. 
Tests involving only two or three samples may reveal gross 
errors in hardware design or manufacturing processes, but 
when relied on for anything more, the conclusions become 
risky (refs. 6-8 and 6-9). 

Attribute tests can be useful in testing for reliability when 
a sufficient sample size is used. For example, 10 samples tested 
without failure statistically demonstrate a population reliability 
of 0.79 at 90-percent confidence; 100 tests without failure 
demonstrate a population reliability of 0.976 at 90-percent 
confidence. To understand better the application of attribute 
tests and the use of figure A-4, consider the following 
examples: 

Example 1: During the flight testing of 50 missiles, five 
failures are observed. What confidence do we have that the 
missile is 80 percent reliable? 

Solute- From figure A-4(f) the answer is read directly 
to be a v r cent confidence level. The a posteriori reliability 
of these 5\ missiles, or that derived from the observed facts, 
is still 45/50 = 90 percent. Thus, future flights will be at least 
80 percent reliable with a 5-percent risk of being wrong. 

Example 2: An explosive switch has a reliability requirement 
of 0.98. How many switches must be fired without a failure 
to demonstrate this reliability at 80-percent confidence? 

Solution 2: From figure A-4(a) the answer is read directly 
as 80 switches. 

Example 3: a test report states that the reliability of a device 
was estimated to be 0.992 at 95-percent confidence based on 
a test of 1000 samples. How many failures were observed? 

Solution 3: In figure A-4(d) the 95-percent confidence curve 
crosses the 1000-event line at R = 0.992. Therefore, three 
failures were observed. 

In these examples the population reliability estimates may 
represent any of the P terms or the K factors in the expression 
for product reliability, depending on the definition of failure 
used to judge the test results. For a device that is judged only 
on its capability to remain within certain tolerances, the 
reliability would be the P, term. Had catastrophic failures 
been included, we would have demonstrated the P,P, terms. 



In general, attribute tests include all failure modes as part 
the failure definition and, consequently, the associated 
reliability is product reliability with both the P terms and the 
K factors included. 

Attribute test /safety margin slide rule.— A special-purpose 
slide rule has been developed to facilitate determining attribute 
tcst/safcty margin confidence levels. A slide rule should be 
in the back of this manual. Take it out and use it as you go 
over the following examples: 

Examples 4 (confidence level for attribute test): Attribute 
tests are tests to success. The objective is for a selected number 
of samples, called tests on the slide rule, to operate successfully 
at some predetermined stress level. Some tests, however, may 
fail. This slide rule handles combinations of up to 1000 tests 
and up to 500 failures. The answer is a direct population 
reliability reading of the untested population at a selected 
confidence level. Six confidence levels from 50 to 90 percent 
are available. (The statistical basis for this rule is the * 2 
approximation of binomial distribution.) 

Example 4a: Fifteen items are tested with one failure 
observed. What is the population reliability at 70-percent 
confidence level? 

Solution 4a: Set one failure on the movable slide above the 
70-percent confidence level index. Read from total number 
of tesfs the tests for a population reliability of 0.85 at 
70-percent confidence level. By setting one failure at 
successive levels of confidence this example gives these 
population reliabilities: 0.710 at 95-percent confidence level, 
0.758 at 90 percent, 0.815 at 80 percent, 0.873 at 60 percent, 
and 0.895 at 50 percent. 

Example 4b: A population reliability of 0.9 at 95-percent 
confidence level is desired. How many tests are required to 
demonstrate this condition? 

Solution 4b: Set zero failures at the 95-percent confidence 
level index. From total number of tests read 29 tests 
directly above 0.90 population reliability. Therefore, 29 tests 
without failure will demonstrate this combination. If, however, 
one failure occurs, set one failure at 95 percent. Then 46 others 
must pass the test successfully. Progressively more observed 
failures such as 10 (set of 10 at 95 percent) require 170 
successes (160 + 10). 

Examples 5 (confidence level for safety margins): Safety 
margin S M indicates the number of standard deviations 
between some preselected reliability boundary R h and the 
mean of the measured sample failure distribution. Thus, 

# ~ E h ) -J- o M , where and are the measured 

r.-f mean and standard deviation of the samples under test. The 
larger the sample size, the more nearly the measured S M 
approaches the safety margin of the untested population S 0 . 
This rule equates for six levels of confidence for sample 
sizes N between 5 and 80. (Statistical basis for this rule: 
noncentral t distribution.) 

Example 5a: Ten items are tested to failure with an observed 
or measured Sy of 5.8. What is the lower expected safety 
margin of the untested population at 90-percent confidence? 


Si*!ution 5a: Set 5.8 on the movable slide at the top window 
for the Sm value. Under N = 10 on the 90-percent window, 
read S& > 3.9. Without moving the slide, for successive 
levels of confidence, 4.45 at 80 percent, 4.85 at 70 percent, 
5.21 at 60 percent, and 5.57 at 50 percent. 

Example 5b: Six samples are available for test. What S M is 
required to demonstrate a population safety margin of 4.0 or 
greater at 90-percent confidence level? 

Solution 5b: Using the 90-percent window, set S D = 4.0 
opposite N = 6. At S M read 7. 1 . Therefore, test results of 7. 1 
or greater will demonstrate S D > 4.0 at a 90-percent confi- 
dence level. If 25 samples are available for test, set S D - 4.0 
opposite N = 25 on the 90-percent window .. An of only 
5.0 or greater would demonstrate 4.0 or greater safety margin 
it 90-percent confidence. 

Sneak circuits. — During attribute testing the flight hardware 
may sometimes not work properly because of a sneak circuit. 
A sneak circuit is defined for both hardware and software as 
follows (ref. 6-10): 

(1) Hardware: A latent condition inherent to the system 
design and independent of component failure that inhibits 
a desired function or initiates an undesired function (path, 
timing, indication, label) 

(2) Software: An unplanned event with no apparent cause- 
and-effect relationship that is not dependent on hardware 
failure and is not detected during a simulated system test 
(path, timing, indication, label) 

Each sneak circuit problem should be analyzed, a cause 
determined, and corrective action implemented and verified. 
References 6-10 to 6-12 give a number of examples on how 
this can be done: 

(1) Reluctant Redstone—making complex circuitry simple 

(2) F-4 example 

(3) Trim motor example 

(4) Software example 

A few minutes spent with one of these references should solve 
any sneak circuit problem. 

Attribute test summary.— In summary, four concepts should 
be kept in mind: 

(1) An attribute test, when conducted with only a few 
samples, is not a satisfactory method of testing for reliability. 
But it can identify gross design and manufacturing problems. 

(2) An attribute test is an adequate method of testing for 
reliability only when sufficient samples are tested to establish 
an acceptable level of statistical confidence. 

(3) Some situations dictate attribute tests or no tests at all 
(e.g., limited availability or the high cost of samples, limited 
time for testing, test levels that exceed the limits of test 
equipment, and the need to use the test samples after testing). 

(4) Confidence, a statistical term that depends on supporting 
statistical data, reflects the amount of risk we arc willing to 
take when stating the reliability of a product. 




Figure 6-2 — Samples lesied lo success at 1.5 times reliability boundary. 



Figure 6-3.— Samples tested to failure at 3 times reliability boundary. 



Test-To-Failure Methods 

Let us return momentarily to the problem of interpreting 
the result of two samples tested to success at a maximum 
anticipated stress, or qualification level. This is the reliability 
boundary R h above which a sample is not required to operate 
or survive. This test result is shown in figure 6-1. 

As indicated earlier, such attribute tests tell only whether 
gross defects exist in the devices tested; they tell nothing about 
similar devices that will not be tested. To obtain better results, 
we can test the two samples at a higher stress level, such as 



1 .5 times the reliability boundary. If both samples pass at this 
level, we will certainly feel more confident that similar devices 
will pass the R b . Statistically, however, we are no better off j 
than before. This result is shown in figure 6-2. j 

We can also continue to increase the stress level until both i 
samples fail. If they fail at the same level, such as three times j 
the R h (as shown in fig. 6-3), we can call the device qualified j 
and infer that all similar devices will survive at stress levels j 
up to the R h . 

But what if one sample fails at 1.2 times the R h and the j 
other at 3.5 times the R h (as shown in fig. 6-4)? What then \ 
could we say about the point at which a third sample might ! 
fail? Would it fail at the R h , at 2 times the R hs or below the 
R b l Clearly, this type of test result casts some doubt upon the 
qualification status of the device even though no failure occurs 
at or below the R h . 

Thus, it is desirable to test enough samples for the failure 
distribution or density function to be established, as shown 
in figure 6-5. Afterwards, we can determine the proportion 
of the product that is expected to fail at or below the R h . We 
do this by applying the density function and the cumulative 
distribution theory discussed in chapter 5. 

This method of testing to determine failure distributions is 
called test to failure. Its purpose is to fail the device under 
test, instead of passing it as in the attribute test. 

Application .— As mentioned before, the purpose of the test- 
to-failure technique is to develop failure distribution for a 
product under one or more types of stress. The results are used 
to calculate the demonstrated reliability of the device for each 
stress. In this case the demonstrated population reliability will 
usually be the P, or P w product reliability term. Before going 
further, however, three terms must be understood. 

Reliability boundary.— The reliability boundary, which is 
the maximum anticipated operating stress level, may be 
represented in two ways: 

(1) As a single point, such as 30 g’s, 125 # F, -25 C F, or 
10 W. When the R b is presented this way. we assume that the 
equipment will be operated at the level indicated 100 percent 
of the time. Because this is usually not done, this method 
represents a worst-case situation. 

(2) As a point in a stress-density function. For example, 
the g force reliability boundary for a missile autopilot during 
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Figure 6-6.— Gravity level during missile flight. 

a flight could be expressed as a 3a limit of a norma! 
distribution— say 29 g's— indicating that a stress of 29 g’s or 
more would be experienced only 0. 14 percent of the time. This 
is shown in figure 6-6. 

This method obviously represents a truer picture than 
method (1) of what stress levels to expect. But this type of 
stress information is usually hard to obtain. Subsequent 
sections demonstrate the difference this method makes in 
design philosophy and the resultant reliability values. 

Failure (or strength) distribution , — The failure density 
function reflects the failure distribution of a device under a 
specific stress (re.'s. 6-8 and 6-9). The data used to develop 
a failure distribution, also called a strength distribution, 
represent failure points obtained through test-to-failure methods. 
Figure 6-7 shows such a distribution for a composition resistor 
at high temperatures, which we interpret just as discussed in 
chapter 5. For example, we can say that 50 percent of the 
resistors will fail at 160 °C and below, 84 percent at 170 °C 
and below, etc. 

Safety margin ,— The safety margin S M of a device is 
defined as the number of standard deviations of the strength 
distribution a v that lie between the reliability boundary and 


the mean strength x s . This is stated mathematically as 



Os 


Thus, S M is the same as the x to value calculated in chapter 5 
from 

x __ Limit - Mean 
a a 

when the limit is R^,. (The minus sign is ignored.) 

As an illustration, assume a reliability boundary of -25 °F 
for a hydraulic system. Through test-to-failure exposure at low 
temperatures we are able to define a failure distribution that 
has a mean of x s = — 37 °F and a standard deviation of 
a, = 4 deg F. The safety margin of the system in reference 
to the -25 °F boundary is given by 

_ _R h -x s -25 -(-37) 12 , 


as shown in figure 6-8. 

Having calculated a safety margin, we can solve for the 
percentage of these systems that will lie above or below the 
reliability boundary. For this we use the technique described 
in chapter 5 under “One-Limit Problems.” In our illustration 
a safety margin of 3 indicates (from table 5-7 in chapter 5) 
that 0.998650 of the systems will not fail until the reliability 
boundary of -25 °F is exceeded. If the failure distribution 
represents an out-of-tolerance condition, the safety margin of 
3 indicates a P, of 0.998650 at low temperatures. 

Test procedure and sample size.— Devices that are not 
automatically destroyed upon being operated are normally not 
expended or destroyed during a functional test. Electronic 
equipment usually falls into this category. For such equipment 
a minimum sample size of five is necessary, with each sample 
being subjected to increasing stress levels until failure occurs 
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Figure 6-7.— Failure, < m strength, distribution of resistor in high temperature. 


Figure 6-8.— Safety margin of device in low temperature. 
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or the limits of the testing facility are reached. In the latter 
case no safety margin calculation is possible because no failures 
are observed. Here, we must rely on intuition in deciding the 
acceptability ot the device. 

Test-to- failure procedure and sample size requirements for 
one-shot devices are different because a onc-shot device is 
normally expended or destroyed during a functional test. 
Ordinarce items such as squib switches fall into this category. 
For suv. ., devices at least 20 samples should be tested, but 30 
to 70 would be more desirable. At least 12 failures should be 
observed during a test. In a typical one-shot test, of which 
there are many variations, a sample is tested at the reliability 
boundary and, if it passes, a new sample is tested at pre- 
determined stress increments until a failure occurs. Then, the 
next sample is tested at one stress increment below the last 
failure. If this sample passes, the stress is increased one 
increment for the next sample. This process, depicted in 

figure 6-9, continues until at least 12 failures have been 
observed. 
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Figure 6-9. -Example of one-shot test-to-failurc procedure. 


Safety margins for single failure modes.— For devices that 
exhibit a single failure mode during a test-to-failure exposure, 
the safety margin and the reliability are calculated by the 
technique just discussed in the definition of safety margin. The 
following examples further illustrate the method and show the 
practical results. 

Example 6: A test was conducted on a vendor’s 0.25- and 
0.50-W film resistors to evaluate their ability to operate 
reliably at their rated power levels. Thirty samples of each 
type were tested by increasing the power dissipation until the 
resistance change exceeded 5 percent. The results are shown 

in figure 6-10, from which the following points are 
noteworthy: 

( 1 ) The mean strength of the 0.25-W resistor was less than 
half the mean strength of the 0.50-W resistor: x 0 25 = 1 19 w 
compared with x 0M = 2.6 W. This was to be expected, since 
the 0.50-W resistor was larger, had more volume, and could 
dissipate more energy. 

(2) The standard deviation of the 0.25-W resistor was almost 
the same as that for the 0.50-W resistor: a 0 25 = 0.272 W; 
co w = 0.332 W. This was also expected 'because both 
resistors were made by the same manufacturer and subjected 
to the same process controls and quality acceptance criteria. 

(3) The 0.50-W resistor, because of its higher mean 
strength, had a safety margin of 6.32 in reference to its rated 
power dissipation of 0.50 W. According to table 5-5, this 
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(a) 0.25-W resistor, .v, = 1,19 W; o, = 0,272 W. 

(b) 0.50-W resistor, .v, = 2.6 W; o, = 0.332 W. 

Figure 6- 10.— Test-to-failure results for 0.25- and 0.50-W resistors. 


.,iM. 




-rv * 





means that only 0.0 9 149 resistors would exceed a 5-percent 
resistance change when applied at 0.50 W. The 0.25-W 
resistor, because of its lower mean strength, had a safety 
margin of only 3.45 in reference to its rated power of 0.25 W. 
According to table 5-5 again, this means that 0.0 3 337 
resistors would exceed a 5-percent resistance change when 
applied at 0.25 W. Derating the 0.25 W to 0. 125 W increased 
the safety margin to 3.92 and decreased the expected number 
of failures to 0.0 4 481. an improvement factor of 7.5. This, 
of course, is the reason for derating components, as discussed 
in chapter 4. Although we have indicated that a safety margin 
of 6.32 has statistical meaning, in practice a population safety 
margin of 5 or higher indicates that the applicable failure mode 
will not occur unless, of course, the strength distribution 
deviates greatly from a normal distribution. 

Example 7: A fiberglass material to be used for a flame 
shield was required to have a flexural strength of 15 000 psi. 
The results of testing 59 samples to failure are presented in 
figure 6-11. The strength distribution of the material was 
calculated to have a mean of 19 900 psi and a standard 
deviation of 4200 psi. The safety margin was then calculated as 

„ 15 000-19 900 


Because, from table 5-7, S M = xjo, = 1.17 indicates that 
87.9 percent of the samples will fail at reliability boundaries 
above 15 000 psi, we can see that 12.1 percent will fail at 
boundaries below 15 000 psi. This analysis is optimistic in that 
1 1/59 = 18.7 percent actually did fail below 15 000 psi. The 
test also shows that the reliability of the flame shield could 
be improved by either selecting another type of material to 
obtain a higher mean strength or changing the fabrication 
processes to reduce the large strength deviation. 

Example 8: Samples of transistors from two vendors were 
tested to failure under high temperatures. Failure was defineu 
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j Figure 6-1 1. -Strength distribution in fiberglass material. ,v, = |<t 900 psi; 

o, = 4200 psi. 
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as any out-of-tolerance parameter. The results, shown in * 

figure 6-12, indicate that vendor B*s materials, design, and 

process control were far superior to vendor A's as revealed 

by the large differences in mean strength and standard 

deviation. With an Sy of 1.41, 7.9 percent of vendor /I s 

transistors would fail at the 74 °C reliability boundary: with 

an S M of 8.27, vendor B’s transistors would not be expected 

to fail at all. It is unlikely that an attribute test would have 

identified the better transistor. 

Example 9: Squib switch samples were tested to failure under 
vibration in accordance with the procedure for testing one- 
shot items. The results are shown in figure 6-13, where the 
mean and standard deviations of the failure distribution have 
been calculated from the failure points observed. As shown. 
x s ~ 14 g’s and <j v = 1 .04 g’s to produce a safety margin of 
3.84 in reference to the reliability boundary of 10 g's. 

The preceding examples have shown how the P, product 
reliability term can be effectively demonstrated through test- 
to-failure methods. This has been the case because each 
example except the squib switch involved a tolerance problem. j 

The examples also show that the K.„ factor plays an important 
role in product reliability and that control over K factors can ! 

ensure a significant increase in reliability. 

Multiple failure modes .— Most products perform more than 
one function and have more than one critical parameter for 

each function. In addition, most products are made up of many j 

types of materials and parts and require many fabrication 
processes during manufacture. It follows then that a product 
can exhibit a variety of failure modes during testing. 

In the conduct of a test to failure each failure mode detected * ; 

must be evaluated individually: that is, a failure distribution 
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Figure 6-12.— Tcst-to-fuilure results for two transistors. 
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must be developed for each failure mode and safety margins 
must be calculated for each individual failure distribution. 
Moreover, as mentioned before, at least five samples or failure 
points are needed to describe each failure mode distribution. 

To see this more clearly, consider the test results shown in 
figure 6-14. Here, each of the three failure modes observed 
is described in terms of its own failure distribution and 
resulting safety margin with reference to the same reliability 
bouiK iry. If these failure modes are independent and each 
represents an out-of-tolerance P t condition, the P t of the test 
device is given by 

^uotai = = 3.5 )P t ' 2 (S M = 2. 1 )P t y (S M = 7.6) 

= (0.9998)(0.982 1)( 1 .00) « 0.9819 

This also shows that the independent evaluation of each failure 
mode identifies the priorities necessary to improve the product. 
For example, the elimination of failure mode 2, either by 



Figure 6- 14. — Test-to-failure results when multiple failure modes are observed. 


increasing P t 2 to 1 or by eliminating the mode altogether, 
increases P Mota) from 0.9819 to 0.9998. 

When stress distribution is known .— When safety margins 
are calculated in reference to a single point or a fixed reliability 
boundary, the resulting reliability estimate is conservative 
because it is assumed that the equipment will always be operated 
at the reliability boundary. As an illustration, figure 6-15 
shows the stress distribution for the operating temperature of 
a device and the maximum anticipated operating limit 
(145 °F), which is given in the device specifications and would 
normally be considered the reliability boundary. 

Figure 6-16 shows the strength distribution of the device 
for high temperatures and also that a safety margin for the 
device, when referenced to the 145 °F reliability boundary, 
is 1 54, or a reliability of 93.8 percent. We krow, however, 
thru the 145 6 F limit is the 3a limit of the stress distribution 
and will occur only 0.135 percent of the time. The question 
is, How does this affect the estimated reliability of the device 
in the temperature environment? 

If we select random values from the stress and strength 
distribution and subtract the stress value from the strength 
value, a positive result indicates a success— the strength 
exceeds the stress. A negative result indicates a failure— the 



Figure 6-15.— Stress distribution for operating temperature, x s » 85 *F; 
« 20 deg F. 
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Figure 6*16.— Strength distribution for operating temperature. x s - 165 *F, 
o v = 13 deg F. 

stress exceeds the strength. With this knowledge we can 
calculate a difference distribution and, through the application 
of the safety margin technique, solve for the probability of 
the strength being greater than the stress (i.e., success). This 
difference distribution is also ? : 3tributed normally and has the 
following parameters: 

^difference = x s “ -^stress 
^difference = “ ^stress) ^ 

From the strength and stress distribution parameters given in 
the preceding example (figs. 6-15 and b-16), 


This 3.33 safety margin gives a reliability of 0.9996 when the 
stress distribution is considered. Comparing this result with 
the estimated reliability of 0.938 when the reliability boundary 
point estimate of 145 °F was used shows the significance of 
knowing the stress distribution whet estimating reliability 
values. 

Confidence levels. —As discussed before, the main objective 
in developing a failure distribution for a device by test-to- 
failure methods is to predict how well a population of like 
devices will perform. Of course, such failure distributions, 
along with the resulting safety margins and reliability 
estimates, are subject to error. Errors result fro.n sample size 
limitations in much the same way that the demonstrated 
reliability varies with sample size in attribute testing. Sped 
fically, the mean and the standard deviations of the strength 
distribution must be adjusted to reflect the sample size used 
in their calculation. Tables A-3 to A-5 in appendix A have 
been developed for this purpose by using the noncentral t 
distribution. Table 6-3 shows the applicable appendix A tables 
for selected confidence levels and sample sizes, and the 
examples that follow illustrate their use. 

Example 10. Upon being tested to failure at high temperatures, 
10 devices were found to have a failure distribution of 
x s — 1 12.7 °C and o s = 16 deg C. The reliability boundary 
was 50 °C. Find the safety margin and reliability demonstrated 
at 90-percent confidence. 

Solution 10. Step 1— Solve first for the ot^ed safety margin. 


^difference 85 — 80 F 

difference = ( 2 O 2 + 13 2 ) 1 ' 2 = 24 deg F 

This distribution is shown in figure 6-17. 

Because positive numbers represent success events, we are 
interested in the area under the difference distribution that 
includes only positive numbers. This can be calculated by using 
zero as the reliability boundary and solving for the safety 
margin from 


5 

o*/ 


0 - 80 
’ 2 \ 


3.33 


c _Rb-x s _50- 112.7 
= = — = 


From table 5-7 the observed reliability is 0.99996. 

Step 2— Now refer to table A-5(a) in appendix A, which deals 
with 90-percent confidence limits for safety margins, and 
follow across to column N = 10, the number of samples. The 
values under the N headings in all of the tables listed in 
table 6-3 represent the observeu safety margins for sample 
sizes as calculated from raw test data. The S M column lists 
corresponding population safety margins for the observed 
safety margins shown under the N headings. Finally, corre- 



Figure 6- 17. — Strength and stress difference distribution. x„ « 80 *F; 
o, » 24 deg F. 


TABLE 6-3.— CONFIDENCE LEVEL TABLES 
FOR VARIOUS SAMPLE SIZES 


Confidence 

level, 

percent 

Sample size 

5-12 

13-20 

21-29 

1 

30-100 

Confidence level tables 

99 

A-3(a) 

A-3(b) 

A-3(c) 

A-3(d' 

95 

A-4(a) 

A-4(b) 

A-4(c) 

A-4(ci; 

90 

A-5(a) 

A-5(b) 

A-5(c) 

A-5(d) 



sponding population reliability estimates are shown under the 
P x headings, which may represent P, or P w as applicable. 

Step 3— Proceed down the N- 10 column to 3.923, the 
observed safety margin derived in step 1 . 

Step 4— Having located S M = 3.923 with 10 samples, follow 
horizontally to the left to find the demonstrated population 
safety margin in the S\i column. This is 2.6. 

Step 5— With a population of 2.6, follow the same line to 
the right to find the population reliability estimate under the 
P x heading. This value is 0.9953. Recall that the observed 
safety margin was 3.923 and the observed reliability, 0.99996. 

Example 11: Twelve gyroscopes were tested to failure by 
using time as a stress to develop a wearout distribution. The 
wearout distribution was found to have a x s of 5000 hours and 
a a s of 840 hours. Find the P M demonstjated at 95-percent 
confidence with a reliability boundary of 1000 hours. 
Solution 11: Step 1— The sample safety margin is 

n 1000 - 5000 , ^ 

S M = = 4.76 


Step 2— The population safety margin at 95-percent confidence 
with a 12-sample safety margin of 4.76 is read directly from 
table A -4(a) to be 3.0. 

Step 3— For a population S M of 3.0, the corresponding P w 
under the P x column is 0.9986. Thereby 99.86 percent of the 
gyroscopes will not wear out before 1000 hours have been 
accumulated. 

Safety factor, - This section is included in the discussion 
of test-to-failure methods because the term “safety factor” 
is often confused with safety margin. It is used widely in 
industry to describe the assurance against failure that is built 
into structural products. There are many definitions for safety 
factor S F , with the mos* common being the ratio of mean 
strength to reliability be mdary; 



When dealing with materials with clearly defined* repeatable, 
and “tight” strength distributions, such as sheet and structural 
steel or aluminum, using S F presents little risk. However, 
when dealing with plastics, fiberglass, and other metal sub- 
stitutes or processes with wide variations in strength or 
repeatability, using S M provides a clearer picture of what is 
happening (fig. 6-18). In most cases, we must know the safety 
margin to understand how accurate the safety factor may be. 

Test-to-failure summary ,— In summary, you should under- 
stand the following concepts about test-to-failure applications: 

(1) Developing a strength distribution through test-to-failure 
methods provides a good estimate of the P t and P w product 



(a) Siruclurc A. 

(b) Structure B. 

Figure 6- 18 — two structures with identical safety factors (S F - 13/10 * 1:3) 
but different safety margins. 


reliability terms without the need for the large samples required 
for attribute tests. 

(2) The results of a test-to-failure exposure of a device can 
be used in predicting the reliability of similar devices that 
cannot or will not be tested. 

(3) Testing to failure provides a means of evaluating the 
failure modes and mechanisms of devices for improvement 
purposes. 
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(4) It allows confidence levels to be applied to the safety 
margins and to the resulting population reliability estimates. 

(5) To know how accurate a safety factor may be, we must 
also know the associated safety margin. 


Life Test Methods 


Chapters 3 and 4 introduced the “bathtub” curve used to 
illustrate how the failure rate of a typical system or complex 
subsystem varies during its operating life. In association with 
this curve we identified three traditional failure rate regions: 
the debugging or burn-in region, the intrinsic-failure-rate 
region, and the wearout region. This curve is presented again 
in figure 6-19, but this time with data that indicate when the 
failure rate regions occur. 

This illustration shows that the greatest reduction in failure 
rate during the debugging or burn-in region (as great as 10 
to !) occurs before 600 to 1200 hours of operation. The curve 
also shows that electronic failure rates continue to decrease 
through as much as 26 000 hours, or 3 years, of continuous 
operation without signs of a wearout region. Items of equip- 
ment with true inherent wear mechanisms usuailv enter the 
wearout region at 3000 or more hours. 

It should be obvious that such data provide valuable 
guidelines for controlling product reliability. They figure 
prominent); in the establishment of burn-in requirements, 
predictions of spare part requirements, and an understanding 
of the need or lack of need for a system overhaul program. 
Such data are obtained through laboratory life tests or from 
the normal operation of a fielded system. In either case collecting 
and assessing life data are vital in testing for reliability. 

Application. —Although life test data are derived basical'y 
for use in evaluating the failure characteristics of a product, 
byproducts of the evaluation may serve many ether purposes. 
Four of the most frequent are 

(1) To serve as acceptance criteria for new hardware For 
example, a product may be subjected to a life test before it 
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F'l’ure 6-20.- Failure rule characteristics of commercial jd electronic subsystem. 
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is accepted for delivery to demonstrate that its failure rate is 
below some predetermined value. Examples of such appli- 
cations are burn-in or debugging tests and group B life tests 
conducted on electronic parts. Some manufacturers of 
communications satellites subject all electronic parts to a 
1200-hour burn-in test and use only the ones that survive. 

(2) To identify product improvement methods. Here, life 
tests serve a dual purpose by providing hardware at essentially 
no cost for physics-of-failure analyses. In turn, these analyses 
identify failure mechanisms and the action needed to reduce 
effectively a product's failure rate. In the past 10 years this 
has resulted in significant part failure rate reductions. In fact, 
the failure rates of some components have been reduced so 
far that accelerated life tests (life tests at elevated stress levels) 
and test-to-failure techniques must be employed to attain 
reliability improvements in a reasonable timeframe. 

(3) To establish preventive maintenance policies. Products 
with known or suspected wear mechanisms are life tested 
to determine when the wearout process will begin to cause 
undesirable failure rate trends. Once the wearout region is 
established for a product, system failures can be reduced by 
implementing a suitable preventive maintenance plan or 
overhaul program. This is effectively illustrated in figure 6-20. 
which shows the failure rate trend in a commercial jet aircraft 
subsystem. Here, the upward trend after 4000 hours of 
operation was revealed to be caused by a servomechanism that 
required lubrication. By establishing a periodic lubrication 
schedule for the mechanism, further failures were eliminated. 
Note that this subsystem also exhibited burn-in and intrinsic- 
failure-rate regions. 

(4) To assess reliability. Here, tests are performed or life 
data collected from fielded systems .o establish whether con- 
tractual reliability requirements are actually being met. In cases 
of noncompliance and when the field failures are analyzed, 
one of the preceding methods is employed to improve the 
product, or else a design change is implemented. The effec- 
tiveness of the corrective action is then evaluated from addi- 
tional life data. Because life-iest-observed failure rates include 
catastrophic, tolerance, wearout, and K factor failures. life 

tests usually demonstrate product reliability. 

Test procedure and sample size. -Conducting a life test is 
fairly straightforward. It involves only the accumulation of 
equipment operating time, Precautions must be taken, how- 
ever, when the test is conducted in a laboratory. Operating 
conditions must include all ot' the factors that affect failure 










rates when the device is operated tactically. Major factors are 
environment, power-on and power-off times, power cycling 
rates, preventive maintenance, operator tasks, and field 
tolerance limits. Ignoring any of these factors may lead to an 
unrealistic failure rate estimate. 

When accelerated life tests are conducted for screening 
purposes, stress levels no greater than the inherent strength 
of the product must be chosen. The inherent strength limit can 
be evaluated through test-to-failure methods before the life tests 
are conducted. 

Experience with nonaccelerated life tests of military standard 
electronic parts for periods as long as 5000 hours indicates 
that an average of one to two failures per 1000 parts can be 
expected. For this reason life tests will not provide good 
reliability estimates at the part level except when quantities 
on the order of 1000 or more parts are available. On the other 
hand, life tests are efficient at the system level with only one 
sample as long as the system is fairly complex (includes several 
thousand parts). 

Life tests intendeo to reveal the wearout characteristics of 
a device may involve as few as five samples, although from 
20 to 30 are more desirable if a good estimate of the wearout 
distribution is to be obtained. 

Analyzing life test dato.— Recall from chapter 3 that an 
empirical definition of mean time between failures (MTBF) 
was given as 



Figure 6-21.— Results of complex electronic system life test. 


MTBF = 


Total test hours 
Total observed failures 


Remember also that, because this expression neglects to show 
when the failures occur, it assumes an intrinsic failure rate 
and therefore an intrinsic mean time between failures, or 
MTBF. The assumption of an intrinsic failure rate may not 
be valid in some cases, but life test results have traditionally 
been reported this way. 

To see this illustrated, consider the results of a 4000-hour 
life test of a complex (47 000 parts) electronic system as shown 
in figure 6-21. This graph plots cumulatively in terms of the 
times the 47 failures are observed, so that the slopes of the 
lines represent the failure rate. The solid line shows the system 
failure rate that resulted from assuming an intrinsic failure rate, 
which was 


Total failures _ 47 
Total operation time 4000 


1 failure, '86 hours 


From the plotted test data, it is obvious that this intrinsic failure 
rate was not a good estimate of what really happened. The 
plotted data indicate that there were two intrinsic-failure-rate 
portions: one from 0 to 1000 hours and the other from 1000 
to 4000 hours. In the 0- to 1000-hour region the actual failure 


rate was 


X = — — = 1 failure/29 hours 
1000 

or about 3 times 'ligher than the total average failure rate of 
1/86 hours; in the 1000- to 4000-hour region the actual failure 
rate was 


12 

X = — — = 1 failure/250 hours 
3000 

or about 2.9 times lower than the average. 

This illustration establishes the ^^sirability of knowing when 
failures occur, not just the number of failures. The results of 
analyzing data by regions can be used to evaluate bum-in and 
spare parts requirements. The burn-in region was identified 
to be from 0 to 1000 hours because after this time the failure 
rate decreased by a factor of 8.6. 

This result also has a significant effect on logistics. For 
example, if we assume that the system will accumulate 
1000 hours per year, we can expect during the first year to 
replace 35 parts 

/ 1 failure . \ 

{ itt * 1000 hours ) 

\29 hours / 










whereas during the next and subsequent years we can expect 
to make only four replacements 


1 failure 
250 hours 


x 1000 hours 


Using the average failure rate of 1 failure/86 hours, we 
would have to plan, however, for 28 replacements every year. 
Obviously, the cost impact of detailed analysis can be 
substantial. 

Running averages. —When system failure rates are irregular 
or when there is need to evaluate the effect of different 
operating conditions on a system, running average analyses 
are useful. This can best be illustrated through the example 
presented in figure 6-22. A 300-hour running average in 
50-hour exposures is shown for a complex system during an 
engineering evaluation test. (Running averages are constructed 
by finding the failure rate for the first 300 hours of operation, 
then dropping the first 50 hours and picking up the 300- to 
350-hour interval and calculating the new 300-hour regional 
failure rate, and then repeating the process by dropping the 
second 50 hours of data and adding the next 50 hours for the 
total test period.) From the resultant curve you can readily 
see (1) the effects of the debugging test, (2) the increase in 
failure rate during the high-temperature test and the decrease 
after that test, (3) another increase during low-temperature 
exposure and the subsequent decrease, (4) a slight increase 
caused by vibration, and (5) a continuously decreasing rate 
as the test progressed. The curve indicates that the system is 
the most sensitive to high temperature and that, because the 
failure rate continued to decrease after high-temperature 


exposure, exposure to high temperatures is an effective way 
to screen defective pans from the system. Because the failure 
rate continued to decrease after the tests were completed, 
neither low temperature nor vibration caused permanent 
damage to the system. 

At the end of the 3000-hour period the failure rate was 3.3 
failures per 1000 hours. This reflected a tenfold decrease from 
the initial failure rate during debugging, typical of the results 
observed for many complex systems. An example of a running 
average failure rate analysis that identifies a system wearout 
region is shown in figure 6-23. The increasing failure rate 
after 3000 hours was caused by relay failures (during approx- 
imately 10 000 cycles of operation). This type of information 
can be used to establish a relay replacement requirement as 
part of a system preventive maintenance plan. 

Confidence levels.— As discussed in chapter 4, failure rates 
are statistical. Consequently, they are subject to confidence 
levels just as attribute and test-to-failure results are influenced 
b v such factors. Confidence levels for intrinsic failure rates 
are calculated by using table A-2 in appendix A. 

To use this table, first calculate the total test hours 
accumulated from 

n 

t = E N,t; 

1=1 

where 

Ni / ,|h unit tested 
tj test time of jV, 
n total units tested 


High 


temperature 



1 2 3x10® 

Operating time, hr 


Figure 6-22.-Running average failure rate analysis of life lest data (300-hr 
running average in 50-hr increments). 



Figure 6-23.— Running average Mure rate analysis of life lest data identifying 
wearout region (600-hr running average in 200-hr increments). 
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Then find under the number of failures observed during the 
test the tolerance factor for the desired confidence level. The 
lower limit for the MTBF at the selectco confidence level is 
then found from 


MTBF 

Tolerance factor 

and the upper limit for failure rate from 

N Tolerance factor 
t 

Example 13: A system was life tested for 3000 hours, during 
which six failures were observed. What is the demonstrated 
80-percent-confidence MTBF? 

Solution 13: Step 1— Solve for the total test hours. 

n 

i = E N,i, - 1 x 3000 = 3000 

i=l 

Step 2— From table A-2 find the tolerance factor for six 
failures at 80-pe.cent confidence to be 9.0. 

Step 3— Solve for the demonstrated MTBF. 

/ 3000 _ , 

MTBF = 333 hours 

Tolerance factor 9 

in contrast to the observed MTBF of 3000/6 = 500 hours. 

Example 14: Had four of the six failures in example 13 been 
observed in the first 1000 hours, what would be the 
demonstrated MTBF at 80-percent confidence in the region 
from 1000 to 3000 hours? 

Solution 14: Step I— The total test time is given as t ~ 2000 
hours. 

Step : -From table A-2 find the tolerance factor for two 
failures at 80-percent confidence to be 4,3. 

Step 3— Find the demonstrated MTBF at 80-percent confidence 
after 1000 to 3000 hours. 

20OQ 

MTBF - — ^ = 465 hours 
4.3 

Example If : It is desired to demonstrate an 80-hour MTBF 
on a computer at 90-percent confidence. How much test time 
is required on one sample if no failures occur? 

Solution 15: Step I— From table A-2 find the tolerance 
factor for no failures at 90-percent confidence to be 2.3. 


Step 2— Because the desired 90-percent‘confidence MTBF is 
given as 80 hours and the tolerance factor is known, calculate 
the total test time required from 

/ - (MTBFKTolerance factor) - (80)(2.3) = 184 hours 

to prove that 184 hours with no failures demonstrates an 
80-hour MTBF at 90-perccnt confidence. 

A good discussion of fixed time and sequential tests is given 
in MIL-STD-78ID (ref. 6-3). 

Life test summary.— In summary, the following concepts 
are reiterated: 

(1) Life tests are performed to evaluate product failure rate 
characteristics. 

(2) If “failures" include all causes of system failure, the 
failure rate of the system is the only true factor available for 
evaluating the system's performance. 

(3) Life tests at the part level require large sample sizes if 
realistic failure rate characteristics are to be identified. 

(4) Laboratory life tests must simulate the major factors that 
influence failure rates in a device during field operations. 

(5) The use of running averages in the analysis of life data 
will identify burn-in and wearout regions if such exit. 

(6) Failure rates are statistics and therefore are subject to 
confidence levels when used in making predictions. 
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Concluding Remarks 

To summarize our discussion of test method, figure 6-24 
is presented to illustrate what might be called a failure surface 
for a typical product. This drawing shows system failure r 
versus operating time and environmental stress. These three 
parameters therefore describe a surface in such a way that, 
given an environmental stress and an operating time, the failure 
rate is a point on the surface. 

Test-to-failure methods generate lines on the surface parallel 
to the stress axis; life tests generate lines on the surface parallel 
to the time axis. Therefore, these tests provide a good descrip- 
tion of the failure surface and, consequently, the reliability 
of a product. 

Attribute tests result only in a point on the surfa i if failures 
occur and a point somewhere within the volume if failures do 
not occur. For this reason attribute testing is the least desirable 
method for ascertaining reliability, as indicated in table 6-1. 
Of course, in the case of missile flights or other events that 
produce go/no-go results, an attribute analysis is the only way 
to determine product reliability. 
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Reliability Training 1 

1 . Seven hydraulic power supplies were tested in a combined high-temperature and vibration test. Outputs 
of six of the seven units tested were within limits. 

a. What is die observed reliability ft of the seven units tested? 

A. 0.825 B. 0.857 C. 0.913 

b. What is the predicted population reliability R at 89-percent confidence? 

A. 0.50 B. 0.75 C. 0.625 

c. How many tests (with one failure already experienced) are needed to demonstrate R = 0.88 at 80-percent 
confidence? 

A. 24 B. 15 C. 30 

2. A vibration test was conducted on 20 autopilot sensing circuits with these results: Mean x, — 7.8 g’s; 
standard deviation o, ■ 1.2 g’s; reliability boundaiy f^ ■ 6 g’s. 

a. What is the observed safety margin S u 7 

A. 2.0 B. 1.0 C. 1.5 

b. What is the observed reliabjility it? 

A. 0.900 B. 0.935 C. 0.962 

c. What is the predicted population safety margin at 80-percent confidence? 

A. 1.19 B. 2.19 C. 3.19 

d. What is the predicted population reliability R at 80-percent confidence? 

A. 0.75 B. 0.95 C. 0.88 

3. Twenty-five low-pressure hydraulic line samples were tested to destruction. These <tes are rated to 
carry 30 psia (f? t ); x, = 31.5 psia; a , * 0.75 psia. 

a. What is the observed S M of these test items? 

A. 1.0 B. 2.0 C. 3.0 

b. What is the predicted population safety margin S M at 90-percent confidence? 

A. 0.95 B. 1.25 C. 1.51 

c. The design requirement calls for an Sm 2 4.0 at 90-percent confidence. Alter discussing the problem 
with the designer, it was learned that the 30-psia rating included a 2.5-psia “pad.” Using die corrected 
R b of 27.5 psia, now what are the S M and So at 90-percent confidence? 

i. Sm (observed) ■ ? 

A. 4.22 B. 5.33 C. 6.44 

ii. S D (predicted) ® ? 

A. 4.275 B. 3.75 C. 4.80 


'Answers in given it the end of this mutual. 
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Chapter 7 

Software Reliability 

Software reliability management is highly dependent on how 
the relationship between quality and reliability is perceived. 
For the purposes of this manual, quality is closely related to 
the process, and reliability is closely related to die product. 
Thus, both span the life cycle. 

Before we can stratify software reliability, the progress of 
hardware reliability should be briefly reviewed. Over the past 
25 years the industry has observed (1) the initial assignment 
of “wizard status” to hardware reliability for theory, model- 
ing, and analysis, (2) the growth of the field, and (3) the final 
establishment of hardware reliability as a science. One of the 
major problems was aligning reliability predictions and field 
performance. Once that was accomplished, the wizard status 
was removed from hardware reliability. The emphasis in 
hardware reliability from now to the year 2000, as discussed 
in chapter 1, will be on system failure modes and effects. 

Software: reliability has reached classification as a science 
for many reasons. The difficulty in assessing software reliabil- 
ity is analogous to the problem of assessing the reliability 
of a new hardware device with unknown reliability charac- 
teristics. The existence of 30 to 50 different software reliability 
models indicates the organization in this area. As discussed 
in chapter 1 , hardware reliability started at a few companies 
and later was focused on by the AGREE reports. The field 
then logically progressed through different models in sequence 
over the years. Along the same lines numerous people and 
companies have simultaneously entered the software reliability 
field in their major areas; namely, cost, complexity, and 
reliability. The difference is that at least 1U0 times as many 
people are now studying software reliability as initially studied 
hardware reliability. The existence of so many models and 
their purports tends to mask the fact that several of these 
models have shown excellent correlations between software 
performance predictions and actual software field performance; 
for instance, the Musa model as applied to communications 
systems and the Xerox model as applied to office copiers. There 
are also reasons for not accepting software reliability as a 
science, and they are briefly discussed here. 

One impediment to the establishment of software reliability 
as a science is die tendency toward programming development 
philosophies such as (1) “do it right the first time” (a 
reliability model is not needed), or (2) “quality is a 


programmer’s development tool”, or (3) “quality is the same 
as reliability and is measured by the number of defects in a 
program and not by its reliability.” All of these philosophies 
tend to eliminate probabilistic measures because the managers 
consider a programmer as a software factory whose quality 
output is controllable, adjustable, or both. In actuality, hard- 
ware design can be controlled for reliability characteristics 
better than software design can. Design philosophy experi- 
ments that failed to enhance hardware reliability are again 
being formulated for software design. (Some of the material 
in this chapter is reprinted with permission from ref. 7-1.) 
Quality and reliability are not the same. Quality is charac- 
teristic and reliability is probabilistic. Our approach draws the 
line between quality and reliability because quality is concerned 
with the development process and reliability is concerned with 
the operating product. Many models have been developed and 
a number of the measurement models show great promise. 
Predictive models have been far less successful partly because 
a data base (such as MIL-HDBK-2 17E (ref. 7-2) for 
hardware) is not yet available for software. Software reliability 
often has to use other methods; it must be concerned with the 
process of software product development. 


Models 

The development of techniques for measuring software 
reliability has been motivated mainly by project managers, who 
need not only ways of estimating the personpower needed to 
develop a software system with a given level of performance, 
but also techniques for determining when this level of perfor- 
mance has been reached. Most software reliability models 
presented to date are still far from satisfying these two needs. 

Most models assume that the software failure rate will be 
proportional to the number of implementation and design 
errors in the system, without taking into account that different 
kinds of errors may contribute differently to the total failure 
rate. Eliminating one significant design error may double the 
mean time to failure, whereas eliminating 10 minor imple- 
mentation errors (bugs) may have no noticeable effect. Even 
assuming that the failure rate is proportional to the number 
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of bugs and design errors in the system, no model considers 
the fact that the failure rate will then be related to the system 
workload. For example, doubling the workload without chang- 
ing the distribution of input data to the system may double 
the failure rate. 

Software reliability models can be roughly grouped into four 
categories: time domain, data domain, axiomatic, and other. 

Yime Domain Models 

Models formulated in the time domain attempt to relate 
software reliability (characterized, for instance, by a mean- 
time-to-failure (MTTF) figure under typical workload con- 
ditions) to the number of bugs present in the software at a given 
time during its development. Typical of this approach are the 
models presented by Shooman (ref. 7-3), Musa (ref. 7-4), 
and Jelinsky and Moranda (ref. 7-5). Removing implemen- 
tation errors should increase MTTF, and correlating bug 
removal history with the time evolution of the MTTF value 
may allow the prediction of when a given MTTF will be 
reached. The main disadvantages of time domain models are 
that bug correction can generate more bugs and that software 
unreliability can be due not only to implementation errors but 
also to design (specification) errors, characterization, and 
simulation during testing of the typical workload. 

The Shooman model (ref. 7-3) attempts to estimate the 
software reliability— that is, the probability that no software 
failure will occur during an operating time interval (0./) — 
from an estimate of the number of errors per machine-language 
instruction present in a software system after T months of 
debugging. The model assumes that at system integration there 
are E, errors present in the system and that the system is 
operated continuously by an exerciser that emulates its real 
use. The hazaid function after T months of debugging is assumed 
to be proportional to the remaining errors in the system. The 
reliability of the software system is then assumed to be 

R(t ) = e ~ CEitJ) 

where E(r,T) is the remaining number of errors in the system 
after T months of debugging and C is a proportionality 
constant. The model provides equations for estimating C and 
E(r,T) from the results of the exerciser and the number of 
errors corrected. 

The Jelinsky-Moranda model (ref. 7-5) is a special case of 
the Shooman model. The additional assumption is made that 
each error discovered is immediately removed, decreasing the 
remaining number of errors by one. Assuming that the amount 
of debugging time between error occurrences has an 
exponential distribution, the density function of the time of 
discovery of the » ,h error, measured from the time of 
discovery of the O' - l) ,h error is 

p(ti) - Mi)e 


where A(i) =f(N - /' + 1 ) and N is the number of errors 
originally present. The model gives the maximum likelihood 
estimates for N and /. 

The Jelinsky-Moranda model has been extended by 
Wolverton and Schick (ref. 7-6). They assume that the error 
rate is proportional not only to the number of errors but also 
to the time spent in debugging, so that the chance of discovery 
increases as time goes on. Thayer, Lipow and Nelson 
(ref. 7-7) give another extension in which more than one error 
can be detected in a time interval, with no correction being 
made after the end of this interval. New maximum likelihood 
estimators of N and / are also given. 

All the models presented so far attempt to predict the 
reliability of a software system after a period of testing and 
debugging. In a good example of an application of this type 
of model, Miyamoto (ref. 7-8) describes the development of 
an on-line, real-time system for which a requirement is that the 
mean time between software errors (MTBSE) has to be longer 
than 30 days. The system will operate on a day-by-day basis, 
13 hours a day. (It will be loaded every morning and reset 
every evening.) The requirement is formulated so that the value 
of the reliability function R(t) for t - 13 hours has to be 
greater than e (-l3/MTBSE) = 0.9672. Miyamoto also gives the 
variations in time of the MTBSE as a function of the debugging 
time. The MTBSE remained low for most of the debugging 
period, jumping to an acceptable level only at the end. The 
correlation coefficient between the remaining number of errors 
in the program and the failure rate was 0.77, but the scatter 
plot shown is disappointing and suggests that the correlation 
coefficient between the failure rate and any c ther system 
variable could have given the same value. In the same paper 
Miyamoto describes in detail how the system was tested. 

None of the models above takes into account that in the 
process of fixing a bug, new errors may be introduced in the 
system. The final number given is usually the mean time 
between software errors, but only Miyamoto points out that 
this number is valid only for a specific set of workload 
conditions. 

Other models for studying the improvement in reliability 
of a software item during its development phase exist, such 
as Littlewood (ref. 7-9), where the execution of a program 
is simulated with continuous-time Markov switching among 
smaller programs. This model also demonstrates that under 
certain conditions in the software system structure, the failure 
process will be asymptotically Poisson. Trivedi and Shooman 
(ref. 7-10) give another Markov model, where the most 
probable number of errors that will have been corrected at 
any time t is based on preliminary modeling of the error 
occurrence and repair rates. The model also predicts the 
system’s availability and reliability at time r. Schneidewind 
(ref. 7-11) describes a model which assumes that the failure 
process is described by a nonhomogeneous Poisson process. 
The rate of error detection in a time interval is assumed to 
be proportional to the number of errors present during that 


interval. This leads to a Poisson distribution with a decreasing 
hazard rate. 


Data Domain Models 


Another approach to software reliability modeling is studying 
the data domain. The first model of this kind is described by 
Nelson (ref. 7-12). In principle, if sets of all input data upon 
which a computer program can operate are identified, the 
reliability of the program can be estimated by running the 
program for a subset of input data. Thayer, Lipow, and Nelson 
(ref. 7-7) describe data domain techniques in more detail. 
Schick and Wolverton (ref. 7-13) compare the time domain 
and data domain models. However, different applications will 
tend to use different subsets of all possible input data, yielding 
different reliability values for the same software system. This 
fact is formally taken into account by Cheung (ref. 7-14), 
where software reliability is estimated from a Markov model 
whose transition probabilities depend on a user profile. Cheung 
and Ramamoorthy (ref. 7-15) give techniques for evaluating 
the transition probabilities for a given profile. 

In the Nelson model (ref. 7-12) a computer program is 
defined as a computable function F defined on the set 
£«(£,, / = 1 /V), where £ includes all possible com- 

binations of input data. Each £/ is a sample of data needed 
to make a run of the program. Execution of a program 
produces, for a given value of £,-, the function value £(£/)• 

In the presence of bugs or design errors a program actually 
implements F ' . Let £, be the set of input data such that F' (£,) 
produces an execution failure (execution terminates prema- 
turely, or fails to terminate, or the results produced are not 
acceptable). If N e is the quantity of £, leading to failure F„ 




is the probability that a run of the program will result in an 
execution failure. Nelson defines the reliability R as the 
probability of no failures or 


/?= 1 -p=\~ 


In addition, this model is further refined to account for the 
fact that the inputs to a program are not selected from £ with 
equal apriori probability but are selected according to some 
operational requirement. This requirement may be charac- 
terized by a probability distribution (P,. / = 1 , . . . ,N) , P, 
being the probability that the selected input is £,. If we define 
the auxiliary variables K, to be 0 if a tun with £, is successful, 
and 1 otherwise. 


P = E P,Y, 

/-l 


where p is again the probability that a run of the program will 
result in an execution failure. 

A mathematical definition of the reliability of a computer 
program is given as the probability of no execution failures 
after n runs. 


*(„)=*" = (1 -p)" 


The model elaborates on how to choose input data values at 
random for £ according to the probability distribution P, to 
obtain an unbiased estimator of R(n). In addition, if the 
execution time for each £, is also known, the reliability 
function can be expressed in terms of the more conventional 
probability of no failure in a time interval (0, t). 

Chapter 6 in Thayer, Lipow, and Nelson (ref. 7-7) extends 
the previous models to take into account how the testing of 
input data sets should be partitioned. Also discussed are the 
uncertainty in predicting reliability values, the effect of 
removing software errors, and the effect of program structure. 


Axiomatic Models 


The third category includes models in which software 
reliability (as well as software quality in general) is postulated 
to obey certain universal laws (Ferdinand and Sutherla, ref. 
7-16; Fitzsimmons and Love, ref. 7-17). Although such 
models have generated great interest, their general validity has 
never been proven and, at most, they only give an estimate 
for the number of bugs present in a program. 

The best-known axiomatic model is the so-called software 
science theory developed by Halstead (see ref. 7-17). Halstead 
used an approach similar to thermodynamics to provide quan- 
titative measures of program level, language level, algorithm 
purity, program clarity, effect of modularization, programming 
effort, and programming time. In particular, the estimated 
number of bugs in a program is given by the expression 




K proportionality constant 

Eq mean number of mental discriminations between 
errors made by programmer 
V volume of algorithm implementation, N log 2 (n) 


where 


N program length 

n size of vocabulary defined by language used 
More specifically, 

N*N| +N 2 
n = n| + 








TABLE 7-1. -CORRELATION OF EXPERIENCE TO 
SOFTWARE BUG PREDICTION BY 
AXIOMATIC MODELS 


Reference 

Correlation coefficient 
between predicted and 
real number of bugs 

Funami and Halstead (ref. 7-33) 

0.98, 0.83, 0.92 

Cornell and Halstead (ref. 7-34) 

0.99 

Fitzsimmons and Love (ref. 7-17): 


System A 

0.81 

System B 

.75 

System C 

.75 

Overall 

.76 


where 

N t total number of occurrences of operators in a 
program 

N 2 total number of occurrences of operands in a 
program 

n ( number of distinct operators appearing in a program 
n 2 number of distinct operands appearing in a program 

and Eo has been empirically estimated as approximately 3000. 

Many publications have either supported or contradicted the 
results proposed by the software science theory, including 
a special issue of the IEEE Transactions on Software 
Engineering (ref. 7-18). Though unconventional, the measures 
proposed by the software science theory are easy to compute, 
and in any case it is an alternative for estimating the number 
of bugs in a software system. Table 7-1 shows a correlation 
coefficient between the real number of bugs found in a software 
project and the number predicted by the software science 
theory for several experiments. There are significant corre- 
lations with error occurrences in the programs, although the 
data reported by Fitzsimmons and Love (ref. 7-17) (obtained 
from three General Electric software development projects 
totaling 166 280 statements) show weaker correlation than the 
original values reported by Halstead. 

Other Models 

The model presented by Costis, Landrault, and Laprie 
(ref. 7-19) is based on the fact that for well-debugged 
programs a software error results from conditions on both the 
input data set and the logical paths encountered. We can then 
consider these events random and independent of the past 
behavior of the system (i.e., with constant failure rate). Also, 
because of their rarity, design errors or bugs may have the 
same effect as transient hardware faults. 

The model is built on the following assumptions: 

(1) The system initially possesses N design errors or bugs 
that can be totally corrected by N interventions of the main- 
tenance team. 


(2) The software failure rate is constant for a given number 
of system design errors. 

(3) The system starts and continues operation until a fault f 
is detected: it then passes to a repair state. If the fault is due 

to a hardware transient, the system is put into operation again (-• 
after a period of time for which the probability density function 
is assumed to be known. If the fault is due to a software failure, 
maintenance takes place, during which the error may be f 
removed, more errors may be introduced, or no modifications 
may be made to the software. 

The model computes the availability of the system as a [ 
function of time by using semi-Markovian theory. That is, the u 
system will make state transitions according to the transition ’ 
probabilities matrix, and the time spent in each state is a 
random variable whose probability density function is either 
assumed to be known or is measurable. The main result 
presented by Costis, Landrault, and Laprie (ref. 7-19) is how 
the availability of the system improves (when all the design 
errors have been removed) as the design errors are being 
removed under some restrictive conditions. They show that 
the minimum availability depends only on the software failure 
rate at system integration, and not on the order of occurrence 
of the different types of design errors. The presence of different 
types of design errors only extends the time necessary to 
approach the asymptotic availability. 

The mathematics of the model is complex, requiring 
numerical computation of inverse Laplace transforms for the 
transition probabilities matrix, and it is not clear that the 1 
parameters needed to simulate a real system accurately can ! 
be easily measured from a real system. 

Finally, some attempts have been made to model fault- 
tolerant software through module duplication (Hecht, 
ref. 7-20) and warnings about how not to measure software 
reliability (Littlewood, ref. 7-21). 

None of the preceding models characterizes system behavior 
accurately enough to give the user a guaranteed level of ! 
performance under general workload conditions. They estimate ! 
the number of bugs present in a program but do not provide ' 
any accurate method of characterizing and measuring oper- j 
ational system unreliability due to software. There is a large ■ 
gap between the variables that can be easily measured in a j 
running system and the number of bugs in its software. Instead, ! 

a cost-effective analysis should allow precise evaluation of f 
software unreliability from variables easily measurable in an j 
operational system, without knowing the details of how the i 
software has been written. 

Trends and Conclusions 

With software reliability being questioned as a science, 
programming process control appears to be the popular answer 
to both software reliability and software quality. Measurements 
of the programming process are supposed to ensure the 
generation of an “error free” programming product, if such 
an achievement is possible. Further, quality and productivity 
measurements combined with select leading process indicators 



are supposed to fulfill the control requirements for developing 
quality software. This so-called answer is similar to a philos- 
ophy that failed in attempts to develop hardware reliability 
control. Reliability should be used to predict field performance. 
Especially with real-time communications and information 
management systems, the field performance requirements 
vastly overshadow the field defect level requirements. How 
can we change the present popular trend (toward programming 
process control) to one that includes a probabilistic reliability 
approach? The answer is not a simple one; these models must 
be finely balanced so that a clear separation of reliability and 
quality can be achieved. 

The trends for reliability tasks in the large-scale in egrated 
circuit (LSI) and very large-scale integrated circuit (VLSI) 
hardware areas are in the failure modes and effects analysis 
and the control of failures. The same emphasis can be placed 
on software (programming bugs or software errors). Once this 
is done, reliability models can reflect system performance due 
to hardware and software “defects” because their frequency 
of occurrence and the effects of their presence in the operation 
will be known. This philosophy focuses on the complete 
elimination of critical defects and the specified tolerance level 
of minor defects. Normally, minor defects are easier to find 
and more numerous than the most critical defects and therefore 
dominate a defect-removal-oriented model. 

We conclude that the proper method for developing quality 
programming products combines quality, reliability, and a 
selective measurements program. In addition, a redirection of 
the programming development process to be based in the future 
on die criticality of defects, their number, and their budgeting 
at die various programming life-cycle phases is the dominant 
requirement. A reliability growth model will monitor and 
control the progress of defect removal for the design phases 
and prove a direct correlator to actual system field perfor- 
mance. With such an approach a system can be placed in 
operation at a customer site at a preselected performance level 
as predicted by the growth model. 

Software 

We have discussed software models before describing 
software for several reasons. The reader should not be biased 
or led to a specific type of software. Few papers on soft- 
ware reliability make a distinction between product software, 
embedded software, applications software, and support soft- 
ware. In addition, the models do not distinguish between 
vendor-acquired software and in-house software and com- 
binations of these. 

Categories of Software 

According to Electronic Design Magazine, the United States 
supports at least SO 000 software houses, each grossing 
approximately $500 000 per year. It is projected that software 
sales in the United States will surpass hardware sales and reach 


the $60 billion range. International competition will eventually 
yield error-free software. 

In-house and vendor-acquired software can be put into four 
categories as follows: 

(1) Product software 

(2) Embedded software 

(3) Applications software 

(4) Support software 

Product software .— This categorization is from the view- 
point of the software specialist. Communications digital 
switching systems software is included as ‘‘product software” 
along with the software for data packet switching systems, text 
systems, etc. 

Embedded software .—' This category of software comprises 
programming systems embedded in physical products to 
control their operational characteristics. Examples of products 
are radar controllers, boiler controls, avionics, and voice 
recognition systems. 

Applications software.— This category of software is usually 
developed to service a company’s internal operations. The 
accounting area of this category covers payroll systems, 
personnel systems, etc. The business area includes reservations 
systems (car, motel), delivery route control, manufacturing 
systems, and on-line agent systems. 

Support software .— This category consists of the software 
tools needed to develop, test, and qualify other software 
products or to aid in engineering design and development. The 
category includes compilers, assemblers, test executives, error 
seeders, and development support systems. 

Vendor-acquired software.— This software can be absorbed 
by the previous four categories and is only presented here for 
clarification. It includes FORTRAN compilers, COBOL 
compilers, assemblers, the UNIX operating system, the 
ORACLE data base system, and application packages. 

Processing Environments 

Software can usually be developed in three ways; namely, 
(1) interactive, (2) batch, and (3) remote job entry. In the oper- 
ational environment the ways expand to include real time. 
Real-time development can be characteristic of both product 
software and embedded software. However, because product 
software and embedded software differ greatly in their require- 
ments and th;ir development productivity and quality method- 
ologies, they should not be combined (e.g., avionics has 
size, weight, and reliability requirements resulting in dense 
software of a type that a communications switching system 
does not have). 

Severity of Software Defects 

We must categorize and weigh the effects of failures. The 
following four-level defect severity classification is presented 
in terms of typical software product areas: 

(1) System unusable (generic: frequent system crashes) 
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(a) Management information system (MIS) software 
defects: inability to generate accounts payable; inability 
to access data base; improper billing 

(b) Computer-aided design (CAD), manufacturing 
(CAM), and engineering (CAE) defects: inability to 
use systems; CAD produces incorrect designs 

(c) Telephone switching defects: frequent service outages; 
loss of emergency communications service 

(d) Data communications defects: loss of one or more 
signaling channels; unrecoverable errors in transmis- 
sion; erratic service 

(e) Military system defects: success of mission jeopar- 
dized; inability to exercise fire control systems; loss 
of electronic countermeasure capabilities 

(f) Space system defects: success of space mission jeop- 
ardized; risk of ground support team or flight 
crew life; loss of critical telemetry information 

(g) Process control defects: waste of labor hours, raw 
materials, or manufactured items; loss of control 
resulting in contamination or severe air and water 
pollution 

(2) Major restrictions (generic: loss of some functions) 

(a) MIS software defects: loss of some ticket reservation 

centers or loss of certain features such as credit card 
verification " 

(b) CAD/CAM/CAE defects: loss of some features in 
computer-aided design such as the update function; 
significant operational restrictions in CAM or CAE 
areas; faults produced for which there is no work- 
around 

(c) Telephone switching defects: loss of full traffic ca- 
pability; loss of billing 

(d) Data communications defects: occasional loss of 
consumer data; inability to operate in degraded 
mode with loss of equipment 

(e) Military system defects: significant operational re- 
strictions; toss of intermediate fast frequency 
function in detection systems; loss of one or more 
antijamming features 

(f) Space system defects: occasional loss of telemetry 
data and communications; significant operational or 
control restrictions 

(g) Process control defects: process cannot consistently 
handk exceptions; inability to complete all process 
contro functions 

(3) Minor n strictions (generic: loss of features; inability to 
effectively modify program) 

(a) MIS software defects: mishandling of records; 
system occasionally cannot handle exceptions 

(b) CAD/CAM/CAE defects: occasional errors produced 
in design system; faults produced for which there 
are workarounds 

(c) Telephone switching defects: loss of some support 
feature, such as cal! forwarding or conferencing 


(d) Data communications defects: occasional inability to 
keep up with data rate or requests; occasional minor 
loss of data transmitted or received 

(e) Military system defects: loss of some operational 
modes such as tracking history, monitor or slave 
model of operation, multiple option selection 

(0 Space system defects: occasional loss of update 
information or frame; occasional loss of subframe 
synchronization or dropouts of some noncritical 
measurements 

(g) Process control defects: problems that require a 
workaround to be implemented; minor reductions in 
rate or throughput; manual intervention at some points 
in the process 

(4) No restrictions (generic: cosmetic; misleading documen- 
tation; inefficient machine/person interface) 

Software Bugs Compared With Software Defects 

Software bugs are not necessarily software defects: the term 
“defect” implies that removal or repair is necessary, and the 
term “bug” implies removal, some degree of correction, or 
a certain level of toleration. A recent example of bug toler- 
ation from the telecommunications industry is contained in 
reference 7-22; “It is not technically or economically feasible 
to detect and fix all software problems in a system as large 
as No. 4 Electronic Switching System (ESS). Consequently, 
a strong emphasis has been placed on making it sufficiently 
tolerant of software errors to provide successful operation 
and fault recovery in an environment containing software 
problems.” 

Various opinions exist in the industry about what consti- 
tutes a software failure. Definitions range from a software 
failure being classed as any software-caused processor re- 
start or memory reload to a complete outage. One argument 
against assigning an MTBF to software-caused processor 
restarts or memory reloads is that, if the system recovers in 
the proper manner by itself, there has not been a software 
failure, only a software fault or the manifestation of a software 
bug. From a systems reliability viewpoint, if the system 
recovers within a reasonable time, the event is not to be classed 
as a software failure. 

Hardware and Software Failures 

Microprocessor-based products have more refined defini- 
tions. Four types of failure may be considered: (1) hardware 
catastrophic, (2) hardware transient, (3) software catastrophic, 
and (4) software transient. In general, the catastrophic failures 
require a physical or remote hardware replacement, a manual 
or remote unit restart, or a software program patch. The 
transient failure categories can result in either restarts or 
reloads for the microprocessor-based systems, subsystems, or 
individual units and may or may not requin) further correction. 
A recent reliability analysis of such a system assigned ratios 



r 1 for these categories. Hardware transient faults were assumed 

!?'. [ to occur at 10 times the hardware catastrophic rate, and 

software transient faults were assumed to occur at 100 to 500 
f \ times the software catastrophic rate. 
i: I The time of day is of great concern in reliability modeling 

> . ; and analysis. Although hardware catastrophic failures occur 

< at any time of the day, they often manifest themselves during 
\ busier system processing times. On the other hand, hardware 
if. | transient failures generally occur during the busy hours as do 
j software transient failures. The availability of restart times is 
f. j also critical and in the example presented in reference 7-23, 

!' ; the system downtime is presented as a function of the MTBF 

1 \ of the software and the reboot time. When a system’s predicted 

t [ reliability is close to the specified reliability, such a sensitivity 
l j analysis must be performed. 

j Reference 7-24 presents a comprehensive summary of 
: j developed models and methods that encompass software life* 
jC ' j cycle costs, productivity, reliability and error analysis, and 
pyL j complexity and the data parameters associated with these 
|v : . ' models and methods. The various models and methods are 
compared in reference 7-24 on a common basis, and the results 
£■: j are presented in matrix form. 

Manifestations of Software Bugs 

Many theories, models, and methods are available for 
quantifying software reliability. Nathan (ref. 7-25) has stated, 
“It is contrary to the definition of reliability to apply reliability 
analysis to a system that never really works. This means that 
the software which still has bugs in it really has never worked 
in the true sense of reliability in the hardware sense." This 
statement agrees with reference 7-22, which says that large, 
complex software programs used in the communications 
industry are usually operating with some software bugs. Thus, 
a reliability analysis of such software is different from a 
reliability analysis of established hardware. Software reliability 
is not alone in the need for establishing qualitative and quanti- 
tative models. Reference 7-26 discusses the "bathtub curve” 
and the effect of recent data on electronic equipment failure 
rate, and reference 7-27 discusses the effects of deferred 
maintenance and nonconstant software and hardware fault rates. 

In the early 1980’s work was done on a combined hardware/ 
software reliability model. Reference 7-28 states, "The use of 
steady-state availability as a reliability/maintainability measure 
is shown to be misleading for systems exhibiting both hardware 
and software faults." The authors develop a theory for com- 
bining well-known hardware and software models in a Markov 
process and they consider the topic of software bugs and errors 
based on their experience in the telecommunications field. To 
synthesize the manifestations of software bugs, we must note 
some of the hardware trends for these systems: 

(1) Hardware transient failures increase as integrated 
circuits become denser. 

(2) Hardware transient failures tend to remain constant or 
increase slightly with time after the "infant mortality” phase. 





(3) Hardware (integrated circuit) catastrophic failures 
decrease with time after the "infant mortality" phase. 

These trends affect the operational software of communications 
systems. If the transient failures increase, the error analysis 
and system security software are called into aciion more often. 
This increases the risk of misprocessing a given transaction 
in the communications system. A decrease in the catastrophic 
failure rate of integrated circuits can be significant, as de- 
scribed in reference 7-! 3, which predicts an order-of- 
magnitude decrease in the failure rate of 4K memory devices 
between the first year and the twentieth year. We also lend 
to oversimplify the actual situations. Even with five vendors 
of these 4K devices, the manufacturing quality control person 
may have to set up different screens to eliminate the defective 
devices from different vendors. Thus, the system software will 
see many different transient memory problems and combi- 
nations of them in operation. 

Central control technology has prevailed in communications 
systems for 25 years. The industry has used many of its old 
modeling tools and applied them directly to distributed control 
structures. Most modeling research was performed on large 
duplex processors. With an evolution through forms of 
multiple duplex processors and load-sharing processors and 
on to the present forms of distributed processing architectures, 
the modeling tools need to be verified. With fully distributed 
control systems the software reliability model must be con- 
ceptually matched to the software design in order to achieve 
valid predictions of reliability. 

The following trends can be formulated for software 
transient failures: 

(1) Software transient failures decrease as the system 
architecture approaches a fully distributed control structure. 

(2) Software transient failures increase as the processing 
window decreases (i.e., le >s time allowed per function, fast 
timing mode entry, removal of error checking, removal of 
system ready checks, etc.) 

A fully distributed control structure can be configured to 
operate as its own error filter. In a hierarchy of processing 
levels each level acts as a barrier to the level below and 
prevents errors or transient faults from propagating through 
the system. Central control structures cannot usually prevent 
this type of error propagation. 

If the interleaving of transaction processes in a software 
program is reduced, such as with a fully distributed control 
architecture, the transaction processes are less likely to fail. 
This is especially true with nonconsistent user interaction as 
experienced in communications systems. Another opinion on 
software transient failures is that the faster a software program 
runs, the more likely it is to cause errors (such as encountered 
in central control architectures). Some genet al statements can 
be formulated: 

(1) In large communications systems software transient 
failures tend to remain constant, and software catastrophic 
failures tend to decrease with time, 
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TABLE 7-2.— CRITICALITY INDEX 


Bug 

manifestation 

rate 

Defect 

removal 

rate 

Level 

of 

criti- 

cality 

Failure type 

Failure characteristic 

4 per day 

1 per month 

5 

Transient 

Errors come and go 

3 per day 

1 per week 

4 

Transient 

Errors are repeated 

2 per week 

1 per month 

3 

Transient or 
catastrophic 

Service is affected 

1 per month 

2 per year 

2 

Transient or 
catastrophic 

System is partially 
down 

1 per two 
years 

1 per year 

1 

Catastrophic 

System stops 


(2) In small communications systems software transient 
failures decrease with time. 

(3) As the size of the software program increases, software 
transient failures decrease and hardware failures increase. 

A “missing link” needs further discussion. Several methods 
can be used to quantify the occurrence of software bugs. 
However, manifestations in the system’s operations are detri- 
mental to the reliability analysis because each manifestation 
could cause a failure event. The key is to categorize levels 
of criticality for bug manifestations and estimate their proba- 
bility of occurrence and their respective distributions. The 
importance of this increases with the distribution of the 
hardware and software. Software reliability is often controlled 
by establishing a software reliability design process. Reference 
7-22 presents techniques for such a design process control. 
The final measure is the system test, which includes the 
evaluation of priority problems and the performance of the 
system while under stress as defined by audits, interrupts, 
reinitialization, and other measurable parameters. The missing 
link in quantifying software bug manifestations needs to be 
found before we can obtain an accurate software reliability 
model for measuring tradeoffs in the design process on a 
predicted performance basis. If a software reliability modeling 
tool could additionally combine the effects of hardware, 
software, and operator faults, it would be a powerful tool for 
making design tradeoff decisions. Table 7-2 is an example 
of the missing link and presents a five-level criticality index 
for defects. Previously, we discussed a four-level defect 
severity classification with level four not causing errors. These 
examples indicate the flexibility of such an approach to 
criticality classification. 

Software reliability measurement and its applications are 
discussed in reference 7-29 for two of the leading software 
reliability models, Musa’s execution time model and 
Littlewood’s Bayesian model. Software reliability measure- 
ment has made substantial progress and continues to progress 
as additional projects collect data. The major hurdle of 
establishing a software reliability measuiement tool for use 
during the requirement stage is under way. 


Comparing references 7-30 and 7-29 yields an insight into 
the different methods of achieving software reliability. The 
method described in reference 7-30 concentrates on the design 
process meeting a present level of reliability or performance 
at the various project design stages. When the system meets 
its final software reliability acceptance criteria, the process 
is complete. Reference 7-29 describes a model that provides 
the design process with a continuous software reliability 
growth prediction. The Musa model can compare simultaneous 
software developments and can be used extensively in making 
design process decisions. An excellent text on software 
reliability based on extensive data gathering was published in 
1987 (ref. 7-31). 

We can choose a decreasing, constant, or increasing soft- 
ware bug removal rate for systems software. Although each 
has its application to special situations and systems, a decreasing 
software bug removal rate will generally be encountered. 
Systems software also has advantages in that certain software 
defects can be temporarily patched and the permanent patch 
postponed to a more appropriate date. Thus, this type of defect 
manifestation is treated in general as one that does not affect 
service, but it snould be included in the overall software quality 
assessment. The missing link concerns software bug mani- 
festations. As described in reference 7-32, until the traditional 
separation of hardware and software systems is overcome in 
the design of large systems, it will be impossible to achieve 
a satisfactory performance benchmark. This indicates that 
software performance modeling has not yet focused on the 
specific causes of software unreliability. 
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Reliability Training 1 


1 . In-house and vendor-acquired software can be classified into what four categories? 

A. Product, embedded, B. Useful, embedded, C. Product, embedded, 
applications, and error- applications, and applications, and support 

free software harmful software software 

2. Name the four categories of software reliability models. 

A. Time domain, data axiom, B. Time domain, data 
corollary, and many domain, axiomatic, 

and other 

3. Can the bug manifestation rate be 

A. Equal to the defect removal rate? 

B. Greater than the defect removal rate? 

C. Less than the defect removal rate? 

D. All of the above? 

4. What are the various software processing environments? 

A. Interactive, batch, remote B. Hyperactive, batch, close job C. Interactive, batch, real job 
job entry, and real time entry, and compressed time entry, and remote time 

5. Name the four levels of severity for software defect categori/ 'iiors. 

A. Generic system, functional, B. System unusable, major C. System unusable, system 
category restrictions, and restrictions, minor restric- crashes, loss of features, 

working tions, and no restrictions and minor bugs 

6. An on-line, real-time system has a mean time between software errors of 15 days. The system 

operates 8 hours per day. What is the value of the reliability function? Use the Miyamoto model. 

A. 0.962 B. 0.999 C. 0.978 

7. Is it always necessary to remove every bug from certain software products? 

A. Yes B. No C. Don't know 

8. Name the four types of hardware and software failure. 

A. Hardware part, hardware B. Hardware plan, hardware 
board, software module, build, software cycle, soft- 

software plan ware type cycle. 


C. Hardware catastrophic, hard- 
ware transient, software cat- 
astrophic. software transient 


C. Time axiom, data domain, 
frequency domain, and 
corollary 


'Answers are given al the end of this manual. 


Chapter 8 

i 

j Software Quality Assurance 


j Concept of Quality 

Let us first look at the concept of quality before going on 
to software quality. The need for quality is universal. The 
| concepts of “zero defects” and “doing it right the first time” 
I have changed our perspective on quality management. We 
changed from measuring defects per unit and acceptable quality 
i levels to monitoring the design and cost reduction processes. 
The present concepts indicate that quality is not free. One 
viewpoint is that a major improvement in quality can be 
; achieved by perfecting the process of developing a product. 
Thus, we would characterize the process, implement factors 
to achieve customer satisfaction, correct defects as soon as 
| possible, and then strive for total quality management. The 
j key to achieving quality appears to have a third major factor 
| in addition to product and process. This third factor is the 
environment. People are important. They make the process 
or the product successful . Figure 8- 1 represents the union of 
these three factors. 

; The term “software quality" is defined and interpreted 
; differently by the many companies involved in producing 
| programming products. To place the subject in perspective, 

| we present principles and definitions for software quality from 
| several source materials: 

(I) The purpose of software quality assurance is to assure 
1 the acquisition of l.igh-quality software products on schedule, 
i within cost, and in compliance with the performance re- 
[ quirements (ref. 8- 1). 

j (2) The developer of a methodology for assessing the qua- 
> lity of a software product must respond to various needs. There 

j can be no single quality metric (ref. 8-2). 

| (3) The process of assessing the quality of a software 

f product begins when specific characteristics and certain of the 
I metrics are selected (ref. 8-3). 

I (4) Software quality can be defined as (a) the totality of 
[ ieatures and characteristics of a software product that bear on 
I its ability to satisfy needs (e.g., conform to specifications), 
j (b) the degree to which software possesses a desired 
j combination of attributes, (c) the degree to which a customer 
| or user perceives that software meets his or her expectations. 

1 and (d) the composite characteristics of software that determine 


the degree to which the software in use will meet the expec- 
tations of the user. 

We can infer from these statements and other source 
materials that software quality metrics (e.g., defects per 1000 
lines of code per programmer year, 70 percent successful test 
cases for the first 4 weeks, and zero major problems at the 
preliminary design review) may vary more than hardware 
quality metrics (e.g.. MTBF or errors per 1000 transactions). 
In addition, software quality management has generally 
focused on the process, and software reliability management 
has focused on the product. Since processes differ for different 
software products, few comparative benchmarks are available. 
For hardware, in general, benchmarks have been available for 
a long time (i.e., MIL-HDBK-217E series (ref. 8-4) for 
reliability). Recently, Rome Air Development Center 
(RADC), the sponsor of MIL-HDBK-2I7E. has sponsored 
a survey of software reliability. It was intended to give 
software quality the same status as hardware quality. 

The next step is to discuss what the process of achieving 
quality in software consists of and how quality management 
is involved. The purpose of quality management for program- 
ming products is to ensure that a preselected software quality 
level has been achieved, on schedule, in a cost-effective 
manner. In developing a quality management system the 
programming product's critical life-cycle phase reviews 
provide the reference base for tracking the achievement of 
quality objectives. The International Electrotechnical 
Commission (IEC) system life-cycle phases presented in their 
guidelines for reliability and maintainability management are 
as follows. 

( 1 ) Concept and definition phase, in which the need for the 
product is decided and its basic requirements defined, usually 
in the form of a product specification, which is agreed upon 
between manufacturer and user. 

(2) Design and development phase, in which the product 
hardware and software are created to perform the functions 
described in the product specification. This phase will normally 
include the assembly and testing of a prototype product under 
laboratory simulated conditions or in actual field trial 
conditions and the formulation o' detailed manufacturing 
specifications and instructions Cor operation and maintenance. 



Figure 8-1.— Quality diagram. 


(3) Manufacturing, installation, and acceptance phase, in 
which the design is put into production. In the case of large, 
complex products die installation Of the product on a particular 
site may be regarded ts an extension of the manufacturing - 
process. This phase will normally conclude with acceptiutce 
testing of the product before it is released to this user. 

(4) Operation and maintenance phase, in which the product 
is operated for the period of its useful life. During this phase, 
essential preventive and corrective maintenance actions are 
taken along with product enhancements, and product per- 
formance is monitored. The usefiil life of a product ends when 
its operation becomes uneconomic because of increasing repair 
costs or other factors or the product becomes technically 
obsolete. 

(5) Disposal phase, in which die product reaches the end 
of its planned useful life or the requirement no longer exists 
for tne product, and it is disposed of, destroyed, or, if 
economically feasible, modernized. 

The quality of the programming product can be controlled 
in the first three life-cycle phases in order to achieve the 
expected level of performance of the final product. Once the 
fourth phase has been entered, the operation and maintenance 
[d w t se, die quality of the software is generally fixed. With these 
five life-cycle phase boundaries in place, we can conceptualize 
what can be implemented as “programming quality 
measurement." If the phases and activities are the X and Y 
coordinates, the individual quality metrics can be placed on 
the Z axis as shown in figure 8-2. 

Without stating the specific activities for each phase, we can 
discuss the generalities of software quality and its cost. The 
cost of implementing quality increases with distance along the 
Xaxis. Activities can be arranged along the Y axis so that the 
cost of quality increases with distance along the Y axis. With 
this arrangement we can establish rigorous quality standards 
for the individual quality metrics as a function of cost effec- 
tiveness (e.g., error seeding— the statistical implanting and 
removal of software defects— may be expensive). Other quality 
metrics (e g., test case effectiveness) may cost significantly 
less and could be selected. 

In general, for a programming product the higher the level 
of quality, the lower the costs of the product’s operation and 
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Figure 8-2.— Programming quality measuiement map. 
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Figure 8-3.— Increasing costs of programming defects. 


maintenance phase. This fact produces an incentive for 
implementing quality metrics in die early design phases. The 
programming industry has traditionally required large 
maintenance organizations to correct programming product 
defects. A typical phase-cost curve presented in figure 8-3 
shows the increased costs of correcting programming defects 
in the later phases of the programming product’s life cycle. 
Note that the vertical axis is nonlinear. 


Software Quality 


The next step is to look at specific software quality items. 
Software quality is defined in reference 8-4 as “the achieve- 
ment of a preselected software quality level within the costs, 
schedule, and productivity boundaries established by manage- 
ment." However, agreement on such a definition is often 
difficult to achieve. In practice, the quality emphasis can 
change with respect to the specific product application environ- 
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ment. Different perspectives of software product quality 
have been presented over the years. However, in todays’ 
literature there is general agreement that the proper quality 
level for a particular software product should be determined 
in the concept and definition phase and that quality managers 
should monitor the project during the remaining life-cycle 
phases in order to ensure the proper quality level. 

The developer of a methodology for assessing the quality 
of a software product must respond to the specific character- 
istics of the product. There can be no single quality metric. 
The process of assessing the quality of a software product 
begins with the selection of specific characteristics, quality 
metrics, and performance criteria. 

The specifics of software quality can now be addressed. 
Several areas of interest are 

(1) Software quality characteristics 

(2) Software quality metrics 

(3) Overall software quality metrics 

(4) Software quality standards 

Areas (1) and (2)are applicable during both the design and 
development phase and the operation and maintenance phase. 
In general, area (2) is used during the design and development 
phase before the acceptance phase for a given software 
product. Each of these four areas is now addressed in detail. 

Software Quality Characteristics 

A software quality characteristic tree is presented in refer- 
ence 8-5. The authors assume that different software products 
require different sets of quality characteristics. A product that 



Figure 8-4.— Minagement's view of quality. 

TABLE 8-1. -APPLICATION-DEPENDENT 
SOFTWARE QUALITY CHARACTERISTICS 


I 

t 

I 

I 


Characteristic 

Application 

Importance 

Maintainability 

Aircraft 

High 


Management information 

Medium 


systems 
Test beds 

Low 

Portability 

Spacecraft 

Low 


Test beds 

High 


has a rigorous constraint on size may sacrifice the main- 
tainability characteristic of the software in order to meet its 
operational program size goals. However, this same product 
may need to be highly portable for use on several different 
processors. In general, the primary software quality charac- 
teristics are 

(1) Maintainability 

(2) Portability 

(3) Reliability 

(4) Testability 

(3) Understandabilit, 

(6) Usability 

(7) Freedom from error 

Management’s view of software quality is the quality charac- 
teristics. Established criteria for these characteristics will 
provide the level of quality desired. The quantitative measures 
(metrics) place the quality at the achieved level. This concept 
is shown in figure 8-4. 

Software quality criteria and metrics are directly related to 
the specific product. Too often, establishing die characteristic 
and the metric in the early life-cycle phases without the proper 
criteria leads to defective software. An example of the 
characteristics and their importance for various applications 
is presented in table 8-1. 

Software Quality Metrics 

The entire area of software measurements and metrics has 
been widely published and discussed. Two textbooks (reft. 
8-6 and 8-7) and die establishment of the Institute for 
Electrical and Electronics Engineers (IEEE) Computer 
Society’s working group on metrics, which has developed a 
guide for software reliability measurement, are three exa pies 
of such activity. Software metrics cannot be developed before 
the cause and effect of a software defect have been established 
for a given product with relation to its product life cycle. 

Table 8-2 is a typical cause-and-effect chart for a software 
product. It includes the process indicator concept. At the 
testing stage of product development die evolution of software 
quality levels can be assessed by characteristics such as free- 
dom from error, successful test case completion, and estimate 
of the software bugs remaining. These process indicators can 
be used to predict slippage of the product delivery date, the 
inability to meet original design goals, etc. 

When the programming product enters the qualification, 
installation, and acceptance phase and continues into the mainte- 
nance and enhancements phase, the concept of performance 
is important in the quality characteristic activity. This concept 
is shown in table 8-3, where the S IEC system life-cycle phases 
have been expanded into 10 software life-cycle phases: 

(1) Conceptual planning phase, in which the functional, 
operational, and economic context of the proposed 
software is understood and documented in a product 
proposal 

(2) Requirements definition phase, in which a product 
proposal is expanded into specific product requirements 
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TABLE 8-2.— MEASUREMENT OF SOFTWARE QUALITY CHARACTERISTICS 


Characteristic 

Software life-cycle phase 

3 

4 

5 

7 

9 

Product 

definition 

Top- 

level 

design 

Detailed 

design 

Testing 

and 

integration 

Maintenance 

and 

enhancements 

Maintainability 

Portability 

Reliability 

Testability: 

Test case completion 
Estimate of bugs 
remaining 
Understandability 
Usability 

Freedom from error 

(a) 

(a) 

(a) 

(a) 

<a) 

1 



•h) 

<1 



(a), <c) 

(a), (c) 

i 



quality charat-ttmi*; tbnttU be measured, 
^'hetc impact of poor quality t» realized. 
c Metnc cw b&c form of proms indicator. 


TABLE 8-3.— MEASUREMENTS AND PROGRAMMING PRODUCT LIFE CYCLE 


System life- 
cycle phase 

Software 
life-cycle phase 

Order of precedence 

Primary 

Secondary 

Concept and 
definition 

Conceptual planning (1) 
Requirements definition (2) 
Product definition (3) 





Quality metrics* 



Design and 
development 

Top-level design (4) 
Detailed design (5) 
Implementation (6) 

Quality metrics 
Quality metrics 
Process indicators 11 

Process indicators 
Process indicators 
Quality metrics 

Manufacturing and 
installation 

Testing and integration (7) 
Qualification, installation, 
and acceptance (8) - 

Process indicators 
Performance measures' 

Performance measures 
Quality metrics 

Operation and 
maintenance 

Maintenance and 
enhancements (9) 

Performance measures 



Disposal 

Disposal (10) 






a Mctrk>— qualitative a»scy>mciM. quantitative prediction, or both, 
^ndkainn— im'mtvby month trac%ing of kc> project pauincierv 
quantitative performance awcvuncnt. 


and the requirements, such as performance and functional 
capabilities, are analyzed and translated into unambiguous 
developer-oriented terms 

(3) Product definition phase, in which software engineering 
principles, technical information, and creativity are used 
to describe the architecture, interfaces, algorithms, and 
data that will satisfy the specified requirements 

(4) Top-level design phase, in which the functional, 
operational, and performance requirements are analyzed 
and designs for system architecture, software architecture, 
interfaces, and data are created and documented to satisfy 
requirements 

(5) Detailed design phase, in which the functional, oper- 
ational, and performance requirements are analyzed and 


designs for system architecture, software architecture, 
components, interfaces, and data are further created, 
documented, and verified to satisfy requirements 

(6) Implementation phase, in which the software product 
is created or implemented from the software design and 
the faults are detected and removed 

(7) Testing and integration phase, in which software 
elements, hardware elements, or both are combined into 
an overall system or an element of a system and the 
elements are tested in an orderly process until the entire 
system has been evaluated, integrated, and tested 

(8) Qualification, installation, and acceptance phase, in 
which a software product is formally tested to ensure the 
customer or customer's representative that the product 
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meets its specified requirements. This phase includes all 
steps necessary to deliver, install, and test a specific release 
of the system software and ^ deliverable documentation. 

(9) Maintenance and enhancements phase, in which the 
product is ready for or serving its designated function, 
is monitored for satisfactory performance, and is modified 
as necessary to correct problems or to respond to changing 
requirements 

(10) Disposal phase, in which the product reaches the end 
of its planned useful life or the requirement no longer 
exists for the product and it is disposed of, destroyed or. 
if economically feasible, modernized 

Overall Software Quality Metrics 

Several overall software quality metrics have been put into 
practice and have effectively indicated software quality. Jones 
(ref. 8-8) presents an overall qualify metric called defect 
removal efficiency. The data collected for the overall qualify 
metric are simplified to the more practical expression of 
“defects per 1000 lines of source code.” 

A second overall qualify metric is based on the concept Of 
quality prisms (refs. 8-9 and 8-10), which considers the extent 


of effort with which a given qualify characteristic has been 
implanted into a product and the degree of effort for quality 
that has occurred in each life-cycle phase. An example of the 
extent and degree of effort is presented in table 8-4 for any 
given quality characteristic. 

As table 8-4 shows, 

(1) Each quality characteristic can have a matrix similar to 
this with a specific quality program tailored to a company's 
products. 

(2) The quality effort is extended to each of the product’s 
life-cycle phases to the degree desired by the company. 

(3) For each levei, as the complexity and difficulty of a 
characteristic requirement increase, the intensify of the test 
and verification program effort increases. 

- (4) This matrix will change for each characteristic in 
accordance with company emphasis. 

(5) Traditionally , the quality levels of a product correspond 
to degrees of effort. However, this matrix extends the effort 
to all-phases of the product’s life cycle. 

As an example of using the matrix shown in table 8-4, a 
characteristic such as reliability may be targeted to reach 
service level 2. Then throughout planning, design, testing^ 



TABLE 8-4.— QUALITY CHARACTERISTIC DEGREE/EXTENT MATRIX 


Product 

Service level 

phase 

0 

1 

2 

3 

4 


Planning 

No activity 

General high level 
required 

Specific detailed 

requirements 

definition 

Highly complex required 
definition and support 
model 

Difficult of complex 
required definition 
and prototype 

E 

X 

t 

e 

n 

t 

0 

f 

e 

f 

f 

0 

r 

t 

Design and 
test 

No activity 

General architecture 
consideration; general 
test and measurement 
program 

Detailed architecture 
structure impact; 
language impact; test 
program extended 

Extensive architecture 
and structure consider- 
ation; tailored language, 
operating system, man- 
machine interface impact, 
etc.; code walkthroughs; 
detailed documentation 

Separate quality teams 
to verif) design; detailed 
test facility; extensive 
qualification test plans 
and procedure 

Integration 
and instal- 
lation 

No activity 

General quality 
management program; 
acceptance test; 
nominal change con- 
trol quality program 

Extensive qualifica- 
tion test plans and 
procedure to verify 
characteristics; above- 
nominal-quality- 
requirement verifica- 
tion testing 

Quality teams formed; 
detailed quality config- 
uration control release 
program; extensive data 
collection, verification, 
and analysis 

Specialized quality inte- 
gration, manufacturing, 
and installation programs 
to ensure achievement of 
quality characteristics by 
separate quality organization 

Service 

No activity 

General quality 
tracking and redesign 
program to achieve 
quality objectives 
and requirements 

Formal data collection 
and analysis program 
to verify quality 
objectives; quality 
redesign effort 

Detailed measurements, 
data analysis, and model- 
ing program to verify high- 
level quality objectives; 
extensive redesign 
to obtain quality 

Extensive measures and 
modeling, vigorous data 
analysis, and specialized 
tests to ensure high-level 
achievement of detailed 
quality requirements; 
extensive change program 


No 

quality 

First level of 
quality 

Second level of 
quality 

Third level of 
quality 

Fourth level of 
quality 



Degree of effort 
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integration, and installation, the reliabUity should achieve at 
least level 2. These indicators are tied to the proper major 
phase review points of a product’s life cycle. For most 
characteristics the planning level should be achieved after the 
preliminary design review (PDR); the design level, after die 
development phase or at the critical design review (CDR); die 
integration level, after integration at the qualification testing; 
and die service level, during die operational service reviews. 

Now quality management can apply this matrix to each 
characteris tic in a manner depending on how critical it is to 
ensure achievement of the characteristic. For example, the 
reliability goal for a key system may be 10 or fewer mishandled 
calls per week, but the reliability goal for a private branch 
exchange (PBX) may be only 5 mishandled calls per month. 
These objectives may cause quality management to define a 
planning 2, design 2, integration 2, and service 2 program 
for the key system and a more demanding planning 4, design 
3, integration 3, and service 3 program for the PBX. 

In this manner die quality characteristics are clearly 
identified by detailed criteria that set the scope of and limit 
die required objectives. Once these objectives are identified, 
a quality program can be determined that defines the specific 
required definition, design, test, and measurement efforts. 
No 1 ™^ are nebulous measurements made against vague 
objectives in the service phase of a product’s life cycle in a 
last-minute attempt to improve quality. 

The program for pursuing quality characteristics must be 
established early. If a particular quality characteristic is not 
pursued to a reasonable extent in the p l a nning and design 
phases, a maximum degree of effort (4) may not realistically 
be achieved in the service phase. Conversely, the more uni- 
formly and consistently a quality characteristic is pursued, the 
more achievable and figuratively stable is the characteristic. 
This is graphically reflected for a single characteristic in figures 
8-5 to 8-7, where the quality item is shown as either stable, 
unstable, or extremely costly to stabilize. 

In figure 8-5 an optimum tradeoff of stability and pro- 
ductivity is portrayed. The base of the prism is secure, 
supporting the platform by properly balancing quality versus 
cost. In figure 8-6 schedule pressures have established an 
nnttnhle prism to support die platform. In this example the 
decision was made to send the product into the field at service 
level 1 even though it initially had reached a more extensive 
degree of quality (3) in the planning phase (considerable effort 
to define quality objectives in the planning phase but no 
followup). Figure 8-7 presents the extremely costly view of 
upgrading a programming product in the field to service level 4 
(after passing the first three phases only to the first degree). 
Note the increasing amount of time and effort to achieve 
service levels 1, 2, or 3. Service level 4 in this example is 
usually extremely difficult and expensive, if not impossible, 
to achieve. The measured productivity of such a product will 
most likely be low. 

An excellent example of the need for this type of quality 
management process occurred many years ago. The lessons 
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Figure 8-6.— Instability due to scheduling decision*. 



Figure 8-7.— Extremely costly programming products. 












still apply today. An automated program was proposed to 
generate, from 160 fields of input data per customer, a 
centralized data base that would control a table-driven wired- 
logic system It was estimated that 13 weeks of design time 
would be required to construct this table generator using a 
nominal amount of computer support time. A representative 
of the design group was assigned to define the input and output 
requirements for the support program and verify its operation. 
The program was initially written in assembly language. It 
was later redesigned and split into three separate programs 
written in a high-level language. These programs could then 
be separately designed, verified, and maintained. The main 
consideration became the verification process. An input and 
output test was written to check the extensive program paths. 
Hie project dragged along for a year as verification testing 
attempted to Meet a 2ero-defect objective (imposed after the 
initial design had been completed). Costs increased and the 
schedule became critical as the customer became impatient 
(fig. 8-7). As the program began to function more success- 
fully, deciding the degree of testing required for verification 
became a serious problem. Confrontation developed between 
die design and marketing departments over the commercial 
release of die program. The testing continued without 
agreement on the required degree of effort. Eventually, die 
customer became disillusioned and turned to another firm to 
provide the table generator. 

Had a clear quality management decision been made in the 
planning phase and tracked throughout the development on 


the degree of error-free “verified” operation, the quality 
characteristic objectives for its design architecture and 
structure, the language required for changes, etc., a more 
realistic projection (and control) of schedule and people could 
have been achieved. Several releases to the customer may have 
been required as the program designs and operation were 
verified to a predetermined extent within the various life-cycle 
phases. Had this procedure been followed, both the customer 
and the supplier would have been more satisfied. 

This example offered an excellent opportunity to first 
determine the type and degree of quality desired. Then 
management could have constructed a quality process, in terms 
of the extent and degree of each desired characteristic, with 
a elastic compromise between the schedule, resources, and 
design activity needed to achieve it. In this case many of the 
“ilities,” such as changeability, usability, maintainability, and 
reliability, were subsequently more critically identified. These 
considerations could have been translated into die initial 
requirements for structural design, program segmentation, 
extensive documentation, and type of language as well as the 
amount of code walkthrough, die number of subfunctional 
tests, the amount of error acceptable at first release, the depth 
of verification reviews, etc. From this form of planning, the 
“quality prisms” could have been established to define the 
extent and degree (such as service level 2, 3, or 4) to which 
each of these characteristics should have been pursued in tains 
of project cost restraints, depending on user willingness to pay 
and wait for a quality product. 



Quality management 

Figure 8-8.— Delicate balance— planning complete. 
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Quality management 


Figure 8-9.— Delicate balance— design and testing complete. 



Quality management 


Figure 8-IO.-Delicate balance-integration and installation complete. 


A figuratively secure prismatic base for the programming 
product is presented in figure 8-5. This security is developed 
through execution of an extensive quality program, as 
progressively shown in figures 8-8 to 8-10. A product’s 
quality objective is usually composed of more than one 
characteristic. Previously, those have tentatively been noted 
as maintainability, portability, reliability, testability, under* 
standability, usability, and freedom from error. Thus, quality 
management can extend the support prismatic structure to a 
greater depth than to just one quality characteristic. In practice, 
seven*! quality prisms will be placed together to achieve a firm 
quality base. 


no 


It may be desirable to have a product developed that has 
reached service level 4 for all of the forementioned quality 
characteristics. However, realistic schedules and productivity 
goals must be considered in terms of cost. These considerations 
establish the need for vigorous quality management over all 
life-cycle phases to selectively balance the various possibilities. 
It would be nonsupportive, expensive, and time consuming 
if quality management established the structural combination 
of individual characteristic quality prisms graphically presented 
in figure 8-11. Unfortunately, this is the case for too many 
products. Quality management would do better to establish 
a more consistent support structure, like that represented in 








P Planning 
D Design and last 
I Integration and installation 
S Service 



Figure 8-1 1.— Example of poor quality management. 


P Planning 
D Design and test 
I Integration and installation 
S Service 



Figure 8-12. — Example of good quality management. 


figure 8-12. The figurative result of this cjnsistent effort is 
shown in the solid cost-effective base of figure 8-13. 

If quality characteristics are established, monitored, meas- 
ured. and verified throughout the life cycle, a realistic balance 
can successfully be achieved between quality costs, schedule, 
and productivity. However, it will require an active quality 
management process to establish and track these indicators. 
An example of such a quality management process matrix is 
presented in table 8-5 to quantify the extent and degree of 
effort needed to achieve a desired level of quality. This table 
can be used as a programming product quality worksheet, as 
well as both the characteristic survey data collection instrument 
and part of the final quality prisms planning document. 

As discussed, a quality management team must establish the 
degree of quality that a particular quality characteristic must 
reach throughout its life cycle. It may use specialized support 
tools, measurement systems, and specific product quality 
standards in pursuing its quality objectives. A point system 
can give a quantitative reference for the pursuit of quality. 
The point system can become the basis for trading time versus 
cost to reach specific quality goals. Of course, a firm’s quality 
management will define their own point system. However, the 



Figure 8-13.— Example of solid quality base. 

following example point system will serve as an illustration 
for discussion purposes. 

If a single characteristic’s quality effort has progressed 
through all four levels, as well as through each level’s 
maximum degree, it has accumulated a maximum of 
4 + 4 + 4+ 4=16 points. If another characteristic’s effort 
has moved through the levels only at one-half of its maximum 
degree, it has accumulated 2+2+2+2=8 points. If it 
reached three-quarters of the maximum degree of effort on 
all levels, it has 3 + 3 + 3 + 3= 12 points. Management can 
now assign a reference value to the pursuit of quality for a 
programming product. This is shown in the simplified example 
in table 8-6. For this example the total is 8 + 12 + 13 = 33 
points out of a possible 16 + 16 + 16 = 48 points, or 69 
percent. (In more general terms, this can also be referred to 
as an overall level 3 quality effort in the 50 to 75 percent 
range.) Note that the real indication of the quality objectives 
will be the magnitude of the XI Y (33/48) values. The greater 
the X and Y values, the deeper the degree to which the 
characteristics have been pursued. The greater the X value, 
the more stable the structure has become and the more quality 
objectives the program ning product has achieved. 

If this type of analysis is can ied over all eight characteristics 
(8 x 16), a maximum of 128 points is possible. Products that 
approach this level of effort will have a considerably more 
stable structure than those that are only based upon a 16-point 
single-character structure. The X percent quality reference 
number should also be qualified by a factor to note how many 
characteristics were actually used. This could be shown as 
69 percent/C3 or 33/48/C3. 

Finally, some characteristics will be more complex and 
require greater costs to achieve than others. Thus, a weighting 

TABLE 8-5.— EXAMPLE OF QUALITY MANAGEMENT 
PROCESS MATRIX 

(Number in circle denotes degree of quality selected by a 
quality management process.) 

>» 

I 

0 

1 

UJ 


♦ 

Degree of quality — — ► 


Product 

Quality characteristic 

phase 

Reliability 

Changeability 

Maintainability 

Planning 

1(2)3 4 

1 2 3® 

1 2®4 

Design and test 

l®3 4 

1 2 3® 

1 2®4 

Integration and 
installation 

l©3 4 

1(2)3 4 

1 2 3© 

Service 

4 

1©3 4 

1 2(3)4 


III 



TABLE 8-6.— EXAMPLE OF PURSUIT OF QUALITY 


Product 

Quality characteristic 

phase 

Reliability 

Changeability 

Maintainability 

Planning 

2 


4 

3 

Design and test 



4 

3 

Integration and 



2 

4 

installation 





Service 



2 

3 

Total points/ 

8/16 

12/16 

13/15 

available points 

(50%) 

(75%) 

(81%) 

Total 

(33/48)/C3, or (69%)/C3 


multiplier (WM) can be used to equalize the quality 
characteristics- Weighting multipliers for the preceding 
example are demonstrated in table 8-7. For this example the 
total of 10 + 28 + 19 = 57 points out of a possible 
20 + 40 + 24 - 84 points is 57/84/C3, or 68 percent/C3. This 
three-part programming quality ratio (e.g. , 57/84/C3) can be 
used for reviewing quality across programming products 
wi thin a corporation as a more quantitative cross reference 
of quality costs to quality objectives. 

A quality management process matrix (table 8-5) has been 
presented for pursuing quality throughout a programming 
product’s life cycle. It relates the pursuit of quality character- 
istics to die planning, design and testing, integration and install- 
ation, and service phases. In practice, actual implementation 
of this approach will require die selection of languages, code 
walkthroughs, type of testing, etc., to be specifically defined 
for reaching service quality level 2, 3, or 4. From this matrix 
the impact on schedule and the cost of quality can be projected 
.and monitored. 

This process will also help management to compare the 
extent and degree of quality for products of competing 
c o m pand or internal corporate divisions. Of course, until such 


TABLE 8-7.— EXAMPLE OF USE OF WEIGHTING 
MULTIPLIERS (WM) 


Product 

Quality characteristic 

phase 

Reliability 

Changesbility 

Maintainability 


Level x WM 

Level x WM 

Level x WM 

Planning 

2 x 1 

4x2 

3x2 

Design and test 

2 x 1 

4x2 

3 x 1.3 

Integration and 

2 x 1 

2x3 

4 X 1 

installation 




Service 

2x2 

2x3 

3 x 1.5 

Total points/ 

10/20 

28/40 

19/24 

avtilabl* points 

(50%) 

(70%) 

(79%) 

Total 

(57/84)/C3, or (68%)/CS 



Figure 8-14.— Relationship of messurements and standards. 


a standard is developed, the quality management team will < 
subjectively assign values and multipliers as noted in table 8-5 
and relate them to their own acceptable degree of documen- 
tation, code walkthrough, module tests, etc. These subjective 
values are extremely useful in establishing individual product 
quality effort goals, by translating the concept of quality prisms 
to planning, design, and test considerations that balance 
schedule and cost against quality objectives. However, man- 
agement will now have a more reasonable opportunity to 
pursue and successfully achieve the extent and degree of 
desired quality for their products. 

The ability to specify an overall software quality metric has 
been addressed. Overall quality measurements can be nor- 
malized, as in the quality prisms concept, for purposes of 
comparison. The quality prisms concept can be used to j 
compare the software of two or more different projects within 
the same company or between different companies even if the 
software products have unique applications or utilize different 
programming languages. Quality prisms can also be used to 
combine hardware quality and software quality into an assess- 
ment of the quality of the whole system. 


Software Quality Standards 


The relation sh ip of software quality standards and software 
quality measurements is depicted in figure 8-14. Measure- 
ments and standards must agree. If a set of quality standards 
is established (i.e., zero defects) and quality measurement 
cannot prove it (i.e., through exhaustive testing, error seeding, 
etc.), the software development project must realistically set 
a goal so that both quality standards and measurements can 
be developed. The IEEE has published many articles on and 
general guides for formulating goal criteria. In addition, many 
technical papers are available on specific goals both on a life- 
cycle basis and on a per-delivered software product basis. 


I 


Concluding Remarks 

This chapter has presented a snapshot of where software 
quality assurance is today and has indicated future directions. 
A base for software quality standardization was issued by the 
rF.p.R (ref. 8-1 1). Research is continuing into the use of overall 
software quality metrics and better software prediction tools 







for determining the defect population. In addition, simulators 
and code generators are being further developed so that high- 
quality software can be produced. 

Several key topics have been discussed: 

(1) Life-cycle phases 

(2) Software quality characteristics 

(3) Software quality metrics 

(4) Overall software quality metrics 

(5) Software quality standards 

In addition, table 8-3 presented the topics 

(6) Process indicators 

(7) Performance measures 

Process indicators are closely tied to the software quality 
effort and some people include them as part of the software 
development effort. In general, there are measures such as 
(1) test cases completed versus test cases planned, and (2) die 
number of lines of code developed versus the number expected. 
Such process indicators can also be rolled up (all software 
development projects added together) to give an indication of 
overall company or corporate progress toward a quality soft- 
ware product. Too often, personnel are moved from one 
project to another and thus the lagging projects improve but 
the leading projects decline in their process indicators. The 
life cycle for programming products, as shown in table 8-3, 
should not be disrupted. 

Performance measures, which include such criteria as die 
percentage of proper transactions, the number of system 
restarts, the number of system reloads, and die percentage of 
uptime, should reflect the user’s viewpoint. Hie concept of 
recently proposed performability (ref. 8-12) combines 
performance and availability from the customer’s perspective. 

In general, the determination of applicable quality measures 
for a given software product development is viewed as a 


specific task of the software quality assurance function. The 
determination of the process indicators and performance 
measures is a task of the software quality standards function. 
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Reliability Training 1 

1. What are the three entities that determine quality software? 

A. Process, material, and vibration 

B. Process, product, and environment 

C. Planning, product, and shock 

D. All of the above 

2. What does software quality consist of? 

A. Various aspects of producing programming products 

B. Bar charts for process control 

C. Statistical analysis of software bugs 

D. All of the above 

3. How is the term “software quality” defined? 

A. To assure the acquisition of high-quality software products on schedule, within cost, and in compliance 
with the performance requirements 

B. To ignore various needs 

C. To develop specifications, develop attributes, perceive customer needs, and meet the user’s expectations 

D. All of the above 

4a. What are the 10 software life-cycle phases? 

A. Conceptual; requirements; product definition; design; implementation; testing; vibration; prototypes; 
installation; and disposal 

B. Planning; definition; design; manufacturing; testing; acceptance; debugging; and repair 

C. Conceptual planning; requirements definition; product definition; top-level design; detailed design; 
implementation; testing and integration; qualification, installation, and acceptance; maintenance 
and enhancements; and disposal 

D. All of the above 

4b. What are the IEC system life-cycle phases? 

A. Concept and research; design and plan; manufacture and debug; operation and maintenance; 
and wearout 

B. Concept and definition; design and development; manufacturing and installation; operation and 
maintenance; and disposal 

C. Research and development; design and breadboard; manufacturing and testing; operation and 

maintenance; and disposal 

D. All of the above 

4c. How can the 10 software life-cycle phases be combined to fit in the IEC system life-cycle phases? 

A. Concept and definition: conceptual planning; requirements definition; and product definition 

B. Design and development: top-level design and detailed design 

C. Manufacturing and installation: implementation; testing and integration; qualification; and 
installation and acceptance 

D. Operations and maintenance: maintenance and enhancement 

E. Disposal: disposal 

F. All of the above 


'Answers are given at the end of this manual. 


5. Can there be different degrees of a quality characteristic for different life-cycle phases? 

A. Yes B. No C. Do not know 

6a. The definition of lack of software quality is 

A. The lack of proper planning in early life-cycle phases 

B. The application of dependent software quality characteristics 

C. Poorly developed software that lacks proper criteria in life-cycle phases 

D. All of the above 

6b. Three example characteristics of software quality are 

A. Testing, integration, and portability 

B. Maintainability, portability, and reliability 

C. Design, implementation, and reliability 

D. All of the above 

7. Seven software quality characteristics are 

A. Maintainability, portability, reliability, testability, understandability, usability, and freedom from error 

B. Planning, definition, reliability, testing, software, hardware, usability 

C. Design, implementation, integration, qualification, acceptance, enhancement, maintenance 

D. All of the above 

8. Management has decided that quality engineering should measure four characteristics of the XYZ software: 
maintainability, portability, reliability, and testability. The desired goals set at the beginning of the program 
by management for the characteristic effort were maintainability, 3.5; portability, 3.0; reliability, 3.9; 
and testability. 3.5. The overall goal was thus 87 percent/C4 for the extent of quality. The 2-year program 
gave the following results: 



Planning 

Design and 
test 

Integration 

Service 

Maintainability 

4.0 

3.5 

3.4 

3.4 

Portability 

4.0 

3.0 

3.1 

3.1 

Reliability 

3.5 

3.6 

3.9 

3.9 

Testability 

4.0 

JA_ 

3.5 

3.6 

Total 

15.5 

13.2 

13.9 

14.0 


a. The actual extent of quality was 

A. (87.5%)/C4 B. (88.4%)/C4 C. (88.8%)/C4 D. None of the above 

b. Have the management objectives been achieved? 












Chapter 9 

j Reliability Management 

i 

j Roots of Reliability Management 

t 

Over the past few years the term "reliability management” 
has been raised to a high level of awareness. Previously, 
the management of reliability was concerned with eliminating 
1 failure by testing to prove reliability, and it generally 
; complemented the design function. Quality management, on 
• the other hand, focused on quality control and generally 
| aligned itself with manufacturing and production. The picture 
' began to change with the focus on customer reliability and 
I quality concerns. Specifically, the usage and standardization 
by companies of reliability growth models established that 
the new concept of reliability management is replacing the 
! old concept of the management of reliability. New stress 
: is b eing placed on enlarging the area of reliability concern 
; to all phases of the life cycle. It is felt that all aspects of 
! manag ement operations and functions must be integrated 
into the reliability concept and program. Thus, reliability in 
the manufacturing or production phase is as important as 
reliability in the design phase (ref. 9-1), as shown in figure 9-1 . 
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Planning a Reliability Management 
Organization 

In p lannin g a reliability management organization the 
reliability function must report to a high enough level to be 
effective. If the reporting level does not involve top manage- 
ment in reliability issues, the reporting level is too low. For 
example, many successful programs today encompass 3 to 6 
hours per month at vice-presidential staff meetings. Each 
company must find the level that makes reliability a real issue 
to be addr r«»d A guide to reliability management is ref- 
erence 9-2. . ., 

A functional organization forms groups performing similar 
generic such as planning, design, testing, and reliability. 
Often this type of organization gets muddled down with too 
many levels of management, and specific product priorities 
are often different in the many task groups. However, many 
benefits accrue from the concentration of talent and constant 
technical peer review. With today’s time-to-market pressures, 
building such a large centralized reliability organization is often 
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Fiture 9-1.— Life cycle reiieblllty growth, with two different puts to flnt customer shipment. 
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not the best choice. The team approach, distributed reliability, 
is often selected over functional organization. 

A team organization forms teams of people often with 
diverse talents and backgrounds. Quality circles and reliability 
circles are based on the same organizational approach. Even 
though peer review is not constantly in place, the cross- 
technology knowledge of today's personnel appears to fully 
compensate for the lack of constant peer review. In the soft- 
ware development world, several types of team organization 
exist. For instance, the first type of typical team organization 
is the project team organization. This is a hierarchical 
organization in which programmers with less experience are 
assigned to work for programmers with more experience. The 
project team organization is designed to fit the company 
organization rather than to fit project requirements. The second 
type is the chief programmer team, which employs a highly 
skilled person who performs most of the programming while 
providing technical direction. A third type is the Weinberg 
programming team, which is composed of groups of 10 or 
fewer programmers with complementary skills. Group 
coasensus and leadership role shifts are characteristic of this 
type of team organization. Each of these team organizations 
has advantages depending on the size of the project, the 
newness of the technology being implemented, etc. 

The fourth type of team organization, matrix organization, 
is a hybrid approach that can be a reliability disaster especially 
if time-to-market pressures exist. Often the technology is 
masked by middle management procedural meetings. The 
matrix organization combines functional talent to put teams 
together. These teams report to one manager. Individual 
contribute rs are added to work on one or more tasks of a given 
project or product development. These projects usually report 
to middle management. 

A fifth possibility is based on the theory stated in reference 
9-3 that reliability is actively pursued by involvement starting 
on the vice-presidential level and is organization wide. This 
new style of reliability involves establishing a reliability 
council, dedicating a full-time diagnostic person or team, and 
generally making an upward change in the reliability reporting 
level. Figure 9-2 presents this concept. The reliability 
council's responsibilities are 



Figure 9-2.— Reliability organization. 


(1) To endorse the annual reliability plan 

(2) To regularly review reliability status 

(3) To approve reliability improvement projects 

(4) To set priorities on resources 

(5) To assign tasks 

(6) To regularly review tasks 

(7) To participate in reliability improvement awards 

The reliability council membership may consist of 

(1) The vice-president of the company or division as chairman 

(2) The vice-president’s staff 

(3) The vice-president's business partners 

(4) The corporate engineering director 

(5) The corporate manufacturing director 

(6) The corporate customer services director 

The diagnostic team’s or person’s functions are 

(1) To review the internal reliability status 

(2) To review reliability as perceived by customers 

(3) To recommend tasks to the reliability council 
('») To diagnose problems 

(5) To design experiments 

(6) To collect and analyze data 

The diagnostic team's or person's concerns include 

(1) Reliability, quality, and statistics 

(2) Engineering and manufacturing engineering 

(3) Product development and process optimization 

(4) Product assembly and test strategies 

(5) Customer perception 

This is a new dynamic approach for establishing reliability 
management at the proper level in a corporation while 
optimizing its effectiveness. 


General Management Considerations 

Program Establishment 

In order to design for successful reliability tnd continue to 
provide customers with a reliable product, the following steps 
are necessary: 

U) Determine the r. liability goals to be met. 

(2) Construct a symbolic representation (e.g.. block 
diag'vn or Petri net. ref. 9-4). 

(3) Determine the logistics support and repair philosophy. 

(4) Select the reliability analysis procedure. 

(5) Select the source or sources of the data for failure rates 
and repair rate 

(6) Determine the failure rates and the repair rates. 

(7) Perform the necessary calculations. 

(8) Validate and verify the reliability. 

(9) Measure reliability until customer shipment. 

This section will address the first three steps in detail. 
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Goals and Objectives 

Goals must be placed into proper perspective. They are often 
examined by using models that the producer develops. 
However, one of the weakest links in the reliability process 
is the modeling. Dr. John D. Spragins. an editor for the IEEE 
Transaction on Computers, places this fact in context (ref. 9-3) 
with the following statement: 

Some standard definitions of reliability or availability, such 
as those based on the probability that all components of a 
system are operational a a given time, can be dismissed as 
irrelevant when studying large telecommunication networks. 
Many telecommunication networks are so large that the 
probability drey are operational according to this criterion may 
be very nearly zero; at least one item of equipment may be 
down essentially all of the time. The typical user, however, 
does pot see this unless be or she happens to be the unlucky 
person whose equipment fails; the system may still operate 
perfectly from this user’s point of view. A more meaningful 
criterion is one based on the reliability seen by typical system 
users. The reliability apparent to system operators is another 
valid, but distinct, criterion. (Since system operator commonly 
consider systems down only after failureshavebeen reported 
to them, and may not hear of short self-clearing outages, their 
esttmau.' of reliability are often higher than the values seen 
by users.) 

Reliability objectives can be defined differently for various 
systems. An example fiom the telecommunications industry 
(ref. 9-5) is presented in table 9-1. We can quantify the 
objectives, for example, for a' private automatic branch 
exchange (PABX) (ref. 9-5) as shown in table 9-2. Table 9-2 
presents the reliability specification for a wide variation of 
PABX sizes (from fewer than 120 lines to over 5000 lines). 

Symbolic Representation 

Chapter 3 presents reliability diagrams, models that are the 
symbolic representations of the analysis. The relationship of 
operation and failures can be represented in these models. 


TABLE 9-1. -RELIABILITY OBJECTIVES FOR 
TELECOMMUNICATIONS INDUSTRY 


Module or system 

Objective 

Telephone instrument 

Mean time between failures 

Electronic key system 

Complete loss of service 
Major toss of service 
Minor loss of service 

PABX 

Complete loss of service 
Major loss of service 
Minor loss of serivee 
Mishandled calls 

Traffic service 

Mishandled calls 

position system (TSPS) 

System outage 

Class 5 office 

System outage 

Class 4 office 

Loss of service 

Class 3 office 

Service degradation 


R e d und ancy (simple and compound) is also discussed in 
chapter 3. Performance estimates and reliability predictions 
are now being performed simultaneously by using symbolic 
modeling concepts' such as Petri nets. 

Twenty-five years ago, Carl Adam Petri published a 
mathematical technique for modeling known as a Petri net. 
A Petri net is a tool for analyzing systems and their projected 
behavior. In 1987, Carl Petri delivered the keynote address 
at the international workshop on Petri nets and performance 
models (ref. 9-7). Many applications were discussed at the 
workshop including the use of timed models for determining 
the expected delay in complex sequences of actions, methods 
used to determine the average data throughput of parallel 
computers, and the average failure rates of fault-tolerant 
computer designs. Correctness analysis and flexible > anu- 
facturing techniques were also described. Timed F-tri >ets 
show promise for analyzing throughput performance in com- 
puter and communications systems. 


TABLE 9-2 . —RELIABILITY SPECIFICATION FOR PABX 





Number of lines 




<120 

200 

400 

600 

800 

1200 

3000 

5000 

Common control performance: 









Mean time between catastrophic 

10 

— 










failures, yr 









System outage time per 20 yr, hr 

— 

— 






1 

1 

l 

Mean time between outages, yr 

— 

— 


— 



>5 

>5 

>5 

Mean time between complete 

5 

10 

40 

40 

40 




losses of service, yr 









Service level: 

Mean time between m^jor losses 

200 

400 

300 

200 

ISO 

365 

365 


of service, days 








Mean time between minor losses 

60 

60 

50 

40 

30 

30 

15 



of service, days 








Degradation of service, hr/yr 

... 

- — 

___ 

___ 


_ 


1 

Mishandled calls, percent 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0,02 
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TABLE 9-3.— SPARES POLICY 

>■. • 
\ * 

r 



Subsystem 

On-site 

spares 

7 

Subdepot 

spares 

7 

Common control and 

Yes 

Yes 

memory 



Network 

No 


Line and trunk units 

Yes 


Peripheral equipment 

No 


Test equipment 

No 

No 


•For replacing spares. 


A Petri net is an abstract and formal graphical model used 
for systems that exhibit concurrent, asynchronous, or non- 
deterininistic behavior. The Petri net model provides accurate 
system information when the model is a valid representation 
of the system and the solution of the model is correct. A Petri 
net is composed of four parts: a set of places, a set of trans- 
itions, an input function, and an output function. The input 
function and the output function relate to transitions and places. 
In general, graphics are used to represent die Petri net 
structures and show the concepts and the problems. A circle 
represents a place, a bar represents a transition, and directed 
arcs connect transitions to places or places to transitions. The 
state of a Petri net is called the PN marking and is defined 
by the number of “tokens” contained in each place. A place 
is an input to a transition if an arc exists from the place to 
the transition and an output if an arc exists from the transition 
to the place. Enabled transitions can be “fired” by removing 
one token from each input place and adding one token to each 
output place. The firing of a transition causes a change of state 
and produces a different PN marking. Reference 9-8 contains 
additional information. Petri nets are a useful reliability 
modeling tool. 


Logistics Support and Repair Philosophy 


The logistics support plan is normally based on criteria such 
as (1) failure rates and repair rates of replaceable units, 
(2) system maturity, (3) whether or not the sites can be served 
by depots or subdepots, and (4) the rate at which additional 
sites are added to the depot responsibility. Since spares are 
the key to support, this chapter will examine them further. 

The size of the spares stock depends on (1) the criticality 
of the replaceable unit to the syster .. (2) the necessary spare 
adequacy level, (3) the number of systems served, 

(4) whether the area served is rural, suburban, or urban, and 

(5) whether the repair facility is on site or remote. A typical 
spares policy for a telecommunications system (ref. 9-9) is 
presented in table 9-3. 

Policies can be formulated for families of systems or for 
multifamily geographical areas. The turnaround time depends 
on the replaceable units failure rate, the repair location, the 


Turnaround 
lime* of 
subdepot 
spates, 
days 


repair costs, etc. A specific spares policy can be tailored to 
a given geographical area. Note that subsystems have differ- 
ent spares policies owing to the criticality of their failures in 
contrast to a blanket spares assignment without regard to 
functionality or survivability. 

Even though the spares location and turnaround tune are 
the same for two different subsystems, the spares adequacy 
can be different. Some spares adequacy levels for a tele- 
communications systems are presented in table 9-4. 

Spares provisioning is an important part of a spares plan. 
Requirements must be clearly stated or they can lead to over- 
or undersparing For example, a spares adequacy of 99.S 
percent can be interpreted in two ways. First, six spares might 
be needed to guarantee that spares are available 99.5 percent 
of the time. Alternatively, if one states that when a failure, 
occurs a spare must be available 99.5 percent of the time, it 
will be necessary to supply 6+1=7 spares. 

The establishment of depot and subdepot sparing, rather than 
only individual site sparing, has proven to be cost effective. 
As an example, table 9-5 presents the depot effectiveness for 
a typical digital PABX. This table indicates that a 14.5-percent 
spares level would be required if only per-site sparing was 
used; however, when one depot serves 100 sites, the required 
spares level is less than 1 percent. 

A centralized maintenance base (CMB) (ref. 9-10) is essential 
to a deferred maintenance concept. Deferred maintenance 
can be available on a real-time basis. When a failure occurs 


TABLE 9-4— SPARES ADEQUACY 


Subsystem 

On-site 

spares? 

Subdepot 

spares 

Depot 

spares 



Adequacy* 

Common control and 

Yes 

0,9995 

0.9995 

memory 

Network 

No 

.995 

.995 

Line and trunk units 

Yes 

.999 

.999 

Peripheral equipment 

No 

.99 

.99 

Test equipment 

No 


.95 


"Probability of having »parc* available, 
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TABLE 9-5.— DEPOT EFFECTIVENESS FOR TYPICAL DIGITAL PABX 


Foreign 

branch 

part 

Control 

automatic 

trunk 

Printed wiring cards for n systems 

Spare printed wiring cards for n systems 

1 

2 

1C 

50 

100 

1 

2 

10 

50 

100 

15002 

6 

65 

130 

650 

3 250 

6 500 

2 

2 

5 

13 

20 

15003 

5 

16 

32 

160 

800 

1600 

1 

I 

2 

5 

7 

15004 

6 

14 

28 

140 

700 

1400 

1 

I 

4 

5 

8 

20703 

8 

28 

56 

280 

1400 

2 80U 

2 

2 

4 

10 

15 

20705 

16 

153 

206 

1530 

7 650 

15 300 

7 

11 

29 

106 

196 

Total 

1058 

2116 

10 580 

52 900 

1US 800 

153 

173 

287 

658 

1001 

Spares, percent of total 

14.5 

8.2 

2.7 

1.2 

0.95 


atan unattended site, the CMB would receive information on 
a display as to the criticality of the failure and the deferred 
maintenance action taken if imposed and would receive a projec- 
tion indicating impending problems. The CMB would analyze 
the situation for the specific site configuration, the processing 
level in the system, and the site’s failure-repair history. 

Input data could consist of items such as the last similar 
occurrence, the next planned visit to the site, the criticality 
of the site to the operating network, the cumulative site failures 
for the last 3 months, and the probability of additional failures 
occurring. The data would be analyzed with a maintenance- 
prediction computer program to generate a table based on 
system loading, such as table 9-6. Often the suggested 
maintenance deferral time is recommended to be the next 
maintenance visit (NMV). The NMV will vary with the 
amount of equipment on site and the projected failure 
frequency (ref. 9-10). 

The combination of deferred maintenance and a centralized 
maintenance base dictates the needs for an efficient spares 
program. Spares planning combined with knowledge of the 
logistics can optimize support costs. A depot stocking plan 
can additionally vary because of many factors, including error 


TABLE 9-$.— MAINTENANCE ACTION 
RECOMMENDATIONS 



Before 

busy 

hour 

Busy 

hour 

After 

busy 

hour 

Off- 

shift 

time 

Repair 

Yes 

Yes 

Yes 

Yes 

Defer repair for (days) 

0 

0 

1 

1 

Is second failure affecting 

No 

Yes 

No 

No 

service? 





Probability of no similar 

0.95 

0.90 

0.82 

0.60 

second failure 





Site failures last month 

Low 

High 

Normal 

Low 

Site failures last year 

Low 

Low 

Normal 

Low 

Transient error rate 

Low 

High 

Low 

Low 


coverage, system maturity, deferred repair, and maintenance 
familiarity. A dynamic (continuously updated) depot stocking 
plan would be cost effective. A dynamic depot model using 
Monte Carlo methods (ref. 9-11) includes unit delivery 
schedules, item usage per month, support personnel efficiency, 
and depot and base repair cycle times. 

Reliability Management Activities 

Performance Requirements 

It is often difficult to translate customer performance 
requirements into design requirements, especially in die area 
of quality and reliability. Reliability encompasses both 
quantitative and qualitative measures. New terms in the 
computer industry, such as “robustness,” are not formally 
metricized. However, we can adapt concepts for the overall 
performance process (ref. 9-12) to apply to reliability as 
presented in figure 9-3. 

If a business’s matrix of reliability requirements is reduced 
to one or more models, subjective and qualitative customer- 
oriented reliability measures can be translated into quantitative 
system-oriented reliability criteria. Figure 9-3 identifies both 
the top-down and bottom-up approaches to reliability 
validation, which . includes (1) translation, (2) allocation, 
(3) requirements, and (4) planning. 

With the identification of the agreed-to system-oriented 
reliability criteria, designer-oriented subsystem or module 
reliability parameters can be allocated as shown in figure 9-3, 
generally by a system reliability team. The team evaluates 
simple versus redundant configurations, levels of fault 
detection and correction implementations, software consid- 
erations, etc. System or module reliability modeling may 
specify reliability requirements for specific components. An 
example of such modeling is a failure modes and effects 
analysis (FMAEA) performed on a product to predict the 
probability of network failures due to a single failure or due 
to a failure after an accumulation of undetected failures. 
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Bottom up 


Top down 



Figure 9-3.— Overall reliability process. 


For example, a replacement product was to uve a veiy large- 
scale integration (VLSI) implementation, and the protection 
against network failures needed to be assessed. An investi- 
gation found no apparent standard industry FMAEA method 
for VLSI components. Because future VLSI products may 
show an increasing need for FMAEA, it is imponant that an 
industry standard be generated. In the network examples 
discussal, a single fault could directly cause a customer- 
oriented problem. 

The bottom 'n approach to reliability validation ensures 
customer satisfaction. The appropriate certification, process 
metrics, and statistical in-process tests must be designed from 
the customer viewpoint. A step-by-step upward certification 
and design review using process metrics can be designed to 
ensure customer-oriented reliability. In addition, we can see 
the need for the independent upward path from reliability 
planning and standards to customer-oriented reliability in 
figure 9-3. This is the key to success, since reliability control 
cannot be bypassed or eliminated from design- or performance- 
related issues. 

Specification Targets 

A system can have a detailed performance or reliability 
specification that is based on customer requirements. The 
survivability of a telecommunications network is defined as 
the ability of the network to perform under stress caused by 
cable cuts or sudden and lengthy traffic overloads and after 
failures including equipment breakdowns. Thus, performance 
and availability have been combined into a unified metric. One 
area of telecommunications where these principles have been 


applied is the design and implementation of fiber-based net- 
works. Reference 9-13 states that “the statistical observation 
that on the average 36 percent of the pairs in a copper cable 
are cut when die cable is dug up, makes the copper network 
‘structurally survivable. ’ ” On the other hand, a fiber network 
can be assumed to be an all or nothing situation with 100 
percent of the circuits being affected by a cable cut, failure, 
etc. In this case study, according to reference 9-13, “cross 
connects and allocatable capacity are utilized by the intelligent 
network operation system to dynamically reconfigure the 
network in the case of failures.” Figure 9-4 (from ref. 9-14) 
presents a concept for specification targets. 

field Studies 

The customer may observe specific results of availability. 
For instance, figure 9-3 has been the basis for the proposal 
of an IEC technology trend document (ref. 9-13). 

System reliability testing is performed today to benchmark 
the reliability, availability, and dependability metrics of complex 
new hardware and software programs. Figure 9-6 (taken from 
ref. 9-1) presents the traditional viewpoint of the design, 
development, and production community on cumulative 
reliability growth. It is possible that the same data generated 
both curves in figure 9-6. When we measure the cumulative 
reliability growth, the decline of production coupled with a 
decline of reliability is masked. If we track the product on 
a quarterly basis, often the product shows a relaxation of proc- 
ess control, incorporation of old, marginal components into 
the last year’s product manufacture, failure to incorporate the 
latest changes into service manuals, knowledgeable personnel 
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transferred to other products, etc. Thus, there is a need to track 
specific products on a quarterly basis (ref. 9-1). 
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Human Reliability 


Analysis Methods 


The major objectives of reliability management are to ensure 
that a selected reliability level for a product can be achieved 
on schedule in a cost-effective manner and that the customer 
perceives the selected reliability level. The current emphasis 
in reliability management is on meeting or exceeding customer 
expectations. We can view this as a challenge, but it should 
be viewed as die bridge between the user and die producer 
or provider. This bridge can be tided “human reliability.” 
In the past, the producer was concerned with the process and 


both. Often there was no correlation between field data, 
die customer’s perception of reliability, and die producer’s 
reliability metrics. Surveys then began to indicate that die 
cu s tomer or user distinguished between reliability perform- 
ance, response to order placement, technical support, service 
quality, etc. 


Human Errors 

Human reliability is defined (ref. 9-16) as “the probability 
of accomplishing a job or task successfully by humans at any 
required stage in system operations within a specified 
minimum time limit (if the time requirement is specified)." 
Although customers generally are not yet requiring human 
reliability models in addition to the requested hardware and 
software reliability models, the science of human reliability 
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Figure 9—7. — Reliability chirectcristk* . 
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is well established. 


Example 

Presendy, the focus in design is shifting from hardware and 
software reliability to human reliability. A recent 2%-year 
study by Bell Communication Research (ref. 9-17) indicated 
that reliability in planning, design, and field maintenance 
procedures must be focused on procedural errors, inadequate 
emergency actions, recovery and diagnostic programs, the 
design of preventive measures to reduce the likelihood of 
procedural errors, and the improvement of the human factors 
in the design and subsequent documentation. The study 
revealed the following results for outages or crashes as shown 
in figure 9-7. Approximately 40 percent of outage events and 
downtime is due to procedural problems (human error). 
In fact, if software recovery problems are included with 
procedural problems, 62 percent of the events and 68 percent 
of the downtime are due to human error. Therefore, human 
reliability planning, modeling, design, and implementation 
must be focused on in order to achieve customer satisfaction. 


Reliability testing usually occurs during product devel- * 
opment and ends with the first product shipment. However, j 
product reliability testing can be cost effectively run through 
the manufacturing life of the product to achieve both continued 
customer satisfaction and the inherent reliability of the product. ^ 

A major concern in planning reliability testing is the maturity j : 
of the specific manufacturing facility. For instance, a new plant ; 
may initially need three to five failures per week of tested j> 
product under controlled test environments in order to shape i 
the manufacturing process and the product specifics. There- j, -1 
fore, detailed failure analysis will be conducted on 150 to 250 | > 
failed items per year. Once plant personnel begin to feel 
comfortable as a team and several of the plant’s processes, 
products, or both are certified, the goal of one failure per week * 

can be instituted in a medium-mature plant. The team in a 
mature plant with few failures can observe leading indicators 

that forewarn of possible problems and can prevent them from 

entering into the shipped product. Thus, in a mature plant the 
goal of one failure per 2 weeks can suffice as a benchmark 
for quality operations to achieve product reliability. 
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Engineering and Manufacturing 

Measuring reliability in a practical way is a challenge. 
Reliability grows with product, process, and customer use 
maturity. We could measure, for example, the reliability at 
first customer shipment and the reliability during a 5-year 
production life. An effective start may be to establish a three- 
to five-level reliability tier concept (ref. 9-18). For example, 
table 9-7 presents a five-tier reliability concept. With this 
concept products can achieve first customer shipment at a mean 
time between failures (MTBF) of 7Tmin). Manufacturing and 
service will accept risks until 7(spec) is reached. Manu- 
facturing has a commitment to drive the MTBF of die product 
up to 7(spec), and engineering has a commitment to provide 
resources for solving desip problems until 7(spec) is reached. 
The qualification team working with this process is now 
involved throughout the desip qualification process through 
field feedback. Ideally, the MTBF’s of tiers 2 to 5 would be 
equal; however, die calibration of reliability modeling tools 
and die accuracy of field MTBF measurements are challenges 
yet to be met in some corporations and industries. Thus, a 
three- to five-tier approach is a practical and effective solution 
for developing reliability measurements. 

Although the MTBF is between 71(min) and 7(spec), 
progress is tracked toward 7(spec) as a goal. The point is to 
find and fix the problems and thus improve the reliability of 
the product. Teamwork and commonality of purpose with 
manufacturing and engineering are necessary in order to deal 
with real problems and not symptoms. After 7(spec) has been 
achieved, an “insurance policy” is necessary to determine if 
anything has gone radically wrong. This can be a gross 
evaluation based on limited data as the “premiums” for a 
perfect “insurance policy” are too high. Once 7(spec) has been 
demonstrated, a trigger can be set at the 50-percent lower 
MTBF limit for control purposes. Improvement plans at this 
level should be based on die return on investment. At maturity, 
Tfintrinsic), dependence on reliability testing can be reduced. 
A few suggestions for reductions are testing fewer samples, 
shortening tests, and skipping testing for 1 or 2 months when 
the personnel feel comfortable with the product or process. 
With a reduced dependence on reliability testing, other 
manufacturing process data can be used for full control. 


TABLE 9-7.— FIVE-TIER RELIABILITY CONCEPT 


Tier 

Mean time 
between 
failures 

Description 

i 

7![min) 

Minimum demonstrated MTBF before shipping 
(statistical test) 

2 

7Tspec) 

Specified MTBF that meets market needs and 
supports service pricing 

3 

7Kdesign) 

Design goal MTBF (calculation) 

4 

^intrinsic) 

Intrinsic MTBF (plant measurement) 

5 

Afield) 

Field MTBF measurement 


User or Customer 

Reliability growth has been studied, modeled, and 
analyzed— usually from the design and development viewpoint. 
Seldom is the process or product studied from the customer’s 
or user’s perspective. Furthermore, the reliability that the first 
customer observes with the first customer shipment can be 
quite different from the reliability that a customer will observe 
with a unit or system produced 5 years later, or last customer 
shipment. Because the customer’s experience can vary with 
the maturity of a system, reliability growth is an important 
concept to customers and should be considered in the 
customer’s purchasing decision. 

The key to reliability growth is the ability to define the goals 
for the product or service from the customer’s perspective 
while reflecting the actual situation in which the customer 
obtains the product or service. For large telecommunications 
switching systems there has been a rule of thumb for 
determining reliability growth. Often systems have been 
allowed to operate at a lower availability than the specified 
availability goal for the first 6 months to 1 year of operation 
(ref. 9- 19). In addition, component part replacement rates have 
often been allowed to be 50 percent higher than specified for 
the first 6 months of operation. These allowances accommo- 
dated craftspersons learning patterns, software patches, design 
errors, etc. 

The key to reliability growth is to have the growth meas- 
urement encompass the entire life cycle of the product. The 
concept is not new, only here the emphasis is placed on the 
customer’s perspective. Reference 9-20 presents the goals of 
software reliability growth (table 9-8). 

Table 9-8 covers a large complex system with built-in fault 
tolerance. Reference 9-21 regarded this system as not 
“technically or economically feasible to detect and fix all 
software problems in a system as large as No. 4 ESS [elec- 
tronic switching system]. Consequently, a strong emphasis has 
been placed on making it sufficiently tolerant of software errors 
to provide successful operation and fault recovery in an envi- 
ronment containing software problems.” 

Reliability growth can be specified from "day 1” on a 
product development and can be measured or controlled on 
a product with a 10-year life until "day 5000.” We can apply 
the philosophy of reliability knowledge generation principles, 
which is to generate reliability knowledge at the earliest 
possible time in the planning process and to add to this base 
for the duration of the product’s useful life. To accurately 
measure and control reliability growth, we must examine the 
entire manufacturing life cycle. One method is the construction 
of a production life-cycle reliability growth chart. 

Table 9-9 presents a chart for setting goals for small (e.g., 
a 60-line PABX or a personal computer), medium, and large 
systems. Small systems must achieve manufacturing, shipping, 
and installation maturity in 3 months in order to gain and keep 
a market share for present and future products. This is 
an achievable but difficult goal to reach. The difference in 
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TABLE 9-8.-1980 GENERIC QUALITY METRICS 
(From reference 9-20. J 

Implementation phase 

Require- Design Laboratory Field test Field 

ments system test performance 


Open questions 

Problems fixed, per 
words 

Problems open, per 
words 

Interrupts, per day 

Audits, per day 

Service affective 
incidents, per 
office month 

Reinitializations, per 
month 

Cutoff calls, per 
10000 

Denied calls, per 
10000 

Trunk out of service, 
min/yr 


TABLE 9-9.— PRODUCTION LIFE-CYCLE RELIABILITY GROWTH CHART 
I Year 


1987 

1988 


1994 ! 

Quarter 



Q1 Q2 Q3 Q4 Q1 Q2 

Small system: 

Reliability growth, 5 0 0 0 0 0 

percent 

Time to steady 3 0 0 0 0 0 

state, months 

Medium system: 100 50 25 10 10 10 

Reliability growth, 
percent 

Time to steady 6 3 2 1 1 1 

state, months 

Large system: 

Reliability growth, 200 100 50 50 33 33 

percent 

Time to steady 12 9 6 3 3 3 

state, months 


Q3 Q4 


10 10 


20 20 






reliability growth characterization between small systems 
and larger systems is that the software-hardware-firmware 
interaction, coupled with the human factors of production, 
installation, and usage, limits the reliability growth over the 
production life cycle for most large, complex systems. 

In certain large telecommunications systems the long 
installation time allows the electronic part reliability to grow 
so that the customer observes the design growth and the 
production growth. Large, complex systems often offer a 
unique environment to each product installation, which dictates 
that a significant reliability growth will occur. Yet, with the 
difference that size and complexity impose on resultant product 
reliability growth, corporations with a wide scope of product 
lines should not present overall reliability growth curves 
on a corporate basis but must present individual product- 
line reliability growth pictures to achieve tond customer 
satisfaction. 
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Reliability Training 1 

1. Reliability management is concerned with what phases of the life cycle? 

A. Design and development B. Manufacturing C. Customer D. All cf the above 

2. Name a new style of organizing reliability activities. 

A. Functional B. Team C. Matrix D. Council 

3. What are the functions of the diagnostic team or person? 

A. Review the internal reliability status 

B. Review reliability as perceived by the customer 

C. Recommend tasks to the reliability council 

D. Diagnose problems 

E. Design experiments 

F. Collect and analyze data 

G. All of the above 

4. Name a goal category for a telephone instrument. 

A. Loss of service 

B. Mean time between failures 

C. Mishandled calls 

D. All of the above 

5. A PABX with 800 lines has a service level reliability specification for the mean time between major 
losses of service (MTBF) of 

A. 150 days B. 1 hour C. 0.1 percent D. All of the above 

6. A Petri net is composed of which of the following parts? 

A. A set of places 

B. A set of transitions 

C. An input function 

D. An output function 

E. All of the above 

7. For a telecommunications system, what is the spares adequacy level for a network subsystem with 
spares depots? 

A. 0.999 B. 0.995 C. 0.95 

8. Turnaround time depends on 

A. Replaceable unit failure rate 

B. Repair location 

C. Repair cost 

D. All of the above 

9. Spares adequacy is the probability of having spares available. 

A. True B. False C. Do not know 


'Answers are given a) the end of this manual. 
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10. What is the normal maintenance action recommendation for the site to defer repair for (days) during 
off-shift time? 

A. 0 B. 2 C. 1 

11. The bottom-up approach to reliability makes use of planning, requirements, allocations, and customer 
orientation. 

A. True B. False C. Do not know 

12. Specification targets can be used to define what performance and availability requirement? 

A. Fully operational 

B. Subliminal availability 

C. Degraded operation 

D. Unusable 

E. Subliminal performance 

F. All of the above 

13. Tracking a product on a quarterly basis often shows 

A. A relaxation of process control 

B. Incorporation of old marginal components 

C. Failure to incorporate the latest changes into service manuals 

D. Knowledgeable personnel transferred to other products 

E. All of the above 

14. If we consider recovery software and procedural problems as human error, human error can account for 
what percentage of outage and downtime problems? 

a. Outage frequency, percent of events/crashes: A. 38 B. 55 C. 62 

b. Downtime (3.5 min), percent per year per machine: A. 42 B. 51 C. 68 

15. Asa benchmark for quality operations to achieve product reliability, what is a reasonable goal (failures 
per week) for a mature plant? 

A. 3.0 B. 1.0 C. 0.5 

16. While the MTBF is between 7(min) and 7(spec), progress is tracked toward what goal? 

A. 7(design) B. 7(spec) C. ^intrinsic) 

17. The key to reliability growth is to have the growth measurement encompass 

A. The design phase 

B. The manufacturing phase 

C. The testing phase 

D. The user phase 

E. The entire life cycle of the product 

18. For a No. 4 ESS system in the field-test phase the number of interrupts per day can be 
A. <20 B. >20 C. 40 

19. An electronic system must achieve manufacturing, shipping, and installation maturity in what period 


of time (months) to gain and keep market share? 

a. Small system: 

A. 1 

B. 2 

C. 3 

b. Medium system: 

A. 4 

B. 6 

C. 12 

c. Large system: 

A. 12 

B. 8 

C. 16 
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Appendix A 

Reliability Information 


The figures and tables in this appendix provide reference 
data to support chapters 2 to 6. For the most part these data 
are self-explanatory. 

Figure A-l contains operating failure rates for military 
standard parts. They relate to electronic, electromechanical, 
and some mechanical parts and are useful in making approx- 
imate reliability predictions as discussed in chapter 3. Their 
use, limitations, and validity are explained in chapter 4. 

Figure A-2 provides failure rate information for making 
approximate reliability predictions for systems that use 
established-reliability parts, such as air- and ground-launched 
vehicles, airborne and critical ground support equipment, 
piloted aircraft, and orbiting satellites. The use of this figure 
is discussed in chapter 4. 

Figure A-3 shows the relationship of operating application 
factor to nonoperating application factor. These data can be 
used to adjust failure rates for the mission condition. The use 
of this figure is also discussed in chapter 4. 

Figure A-4 contains reliability curves for interpreting the 
results of attribute tests. They provide seven confidence levels, 
from SO percent to 99 percent; and six test failure levels, 
from 0 to 5 failures. The use of these figures is discussed in 
chapter S. 

Table A- . contains values of the negative exponential 
function e~ x , where -x varies from 0 to -0.1999. The 
tabulated data make it easy to look up the reliability, where 


the product of failure rate X (or 1/MTBF) and operating time 
t are substituted for —x. Use of this table is discussed in 
chapter 3 and frequently referred to in chapters 4 to 6. 

Table A-2 contains tolerance factors for calculating the 
results of mean-time-be tween-failure tests. It provides seven 
confidence levels, from 50 to 99 percent for 0 to 15 observed 
failures. The use of this table is explained in the table. 
Examples are discussed in chapter 6. 

Tables A-3 to A-5 contain tabulated data for safety margins, 
probability, sample size, and test-demonstrated safety margins 
for tests to failure. They provide three confidence levels, from 
90 to 99 percent, and sample sizes from 5 to 100. Values 
similar to these are presented on the safety margin side of the 
reliability slide rule; the slide rule provides six confidence 
levels and sample sizes from 5 to 80. The use of these tables 
and the slide rule is discussed in chapter 6. 

More information on this subject can be found in references 
A-l and A-2. 
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A-l. Reliability Modeling and Prediction. MIL-STD-756B (plus change 
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TABLE A- 1.— VALUES OF NEGATIVE EXPONENTIAL FUNCTION *-* 


0.0000 

.0001 

.0002 

.0003 

0004 


1. 00000 
.99990 
.99980 
.99970 
.99960 


.0005 

.0006 

.0007 

0008 

.0009 


0.0010 

.0011 

.0012 

.0013 

.0014 


0.0015 

,0016 

.0017 

.0018 

.0019 


0.0020 

.0021 

.0022 

.0023 

.0024 


0.0025 

.0026 

.0027 

.0028 

.0029 


0.0030 

.0031 

.0032 

.0033 

.0034 


0.0035 

.0036 

.0037 

.0038 

.0039 


0.0040 

.0041 

.0042 

.0043 

.0044 


0.0045 

.0046 

.0047 

.0048 

.0049 


0.99950 

.99940 

.99930 

.99920 

.99910 


0.99900 

.99890 

.99880 

.99870 

.99860 


0.99850 

.99840 

.99830 

.99820 

.99810 


0.99800 

.99790 

.99780 

.99770 

.99760 


0.99750 

.99740 

.99730 

.99720 

.99710 


0.99700 

.99690 

.99681 

.99671 

.99661 


0.99651 

.99641 

.99631 

.99621 

.99611 


0.99601 

.99591 

.99581 

.99571 

.99561 


0.0050 

.0051 

.0052 

.0053 

.0054 


0.0055 

.0056 

.0057 

.0058 

.0059 


0.0060 

.0061 

.0062 

.0063 

.0064 


0.0065 

.0066 

.0067 

.0068 

.0069 


0.0070 

.0071 

.0072 

.0073 

.0074 


0.0075 

.0076 

.0077 

.0078 

.0079 


0.99501 

.99491 

.99481 

.99471 

.99461 


0,99452 

.99442 

.99432 

.99422 

.99412 


0.99402 

.99392 

.99382 

.99372 

.99362 


0.99352 

.99342 

.99332 

.99322 

.99312 


0.99302 

.99293 

.99283 

.99273 

.99263 


0.0080 

.0081 

.0082 

.0083 

.0084 


0.99551 

.99541 

.99531 

.99521 

.99511 


0.0085 

.0086 

,0087 

.0088 

0089 


0.99253 

.99243 

.99233 

.99223 

.99213 


0.0100 

0101 

0102 

.0103 

0104 


0.0110 

0111 

.0112 

0113 

.0114 


0.99203 

.99193 

.99183 

.99173 

.99164 


0.99154 

.99144 

.99134 

.99124 

.99114 


0.0090 

.0091 

.0092 

.0093 

.0094 


0.0095 

.0096 

,0097 

.0098 

.0099 


0.99104 

.99094 

.99084 

.99074 

.99064 


0.99054 

.99045 

.99035 

.99025 

.99015 


0105 

0106 
.0107 
0108 
0109 


.0115 

.0116 

.0117 

.0118 

.0119 


0.0120 

.0121 

.0122 

.0123 

.0124 


.0126 

.0127 

.0128 

.0129 


0.99005 

.98995 

.98985 

.98975 

.98965 


0.98955 

.98946 

.98936 

.98926 

.98916 


0.98906 

.98896 

.98886 

.98876 

.98866 


0.98857 

.98847 

.98837 

.98827 

.98817 


0.98807 

.98797 

.98787 

.98777 

.98767 


0.0125 0.98757 


0.0130 

.0131 

.0132 

.0133 

,0134 


.98747 

.98738 

.98728 

.98718 


0.0135 

.0136 

.0137 

.0138 

.0139 


0.0140 

.0141 

.0142 

.0143 

.0144 


0.98708 

.98699 

.98689 

.98679 

.98669 


0.0150 

.0151 

.0152 

.0153 

.0154 


0.0155 

.0156 

.0157 

.0158 

.0159 


0.0160 

.0161 

.0162 

.0163 

.0164 


0.0165 

.0166 

.0167 

.0168 

.0169 


0.0170 

.0171 

.0172 

.0173 

.0174 


0.98659 

.98649 

.98639 

.98629 

.98620 


0.0145 

.0146 

.0147 

.0148 

,0149 


0,98610 

.98600 

.98590 

.98580 

.98570 


0.0175 

.0176 

.0177 

.0178 

.0179 


0.98511 

.98501 

.98491 

.98482 

.98472 


0.98462 

.98452 

.98442 

.98432 

.98423 


0.98413 

.98403 

.98393 

.98383 

.98373 


0.98364 

.98354 

.98344 

.98334 

.98324 


0 98314 
.98305 
.98295 
.98285 
.98275 


0.0180 

.0181 

.0182 

.0183 

.0184 


0.98265 

.98255 

.98246 

.98236 

.98226 


0.0200 

.0201 

,0202 

.0203 

.0204 


0.98216 

.98206 

.98196 

.98187 

.98177 


0.0185 

.0186 

.0187 

.0188 

.0189 


0.98560 

,98551 

.98541 

.98531 

.98521 


0.0190 

.0191 

.0192 

.0193 

.0194 


0.0195 

.0196 

.0197 

.0198 

.0199 


0.98167 

.98157 

.98147 

.98138 

.98128 


0.98118 

.98108 


.98089 

,98079 


0.98069 

.98059 

.98049 

.98039 

.98030 
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0.0205 

.0206 

.0207 

.0208 

.0209 


0.0210 

.0211 

.0212 

.0213 

.0214 


0.0215 

.0216 

.0217 

.0218 

.0219 


0.0220 

.0221 

.0222 

.0223 

.0224 


0.0225 

.0226 

.0227 

.0228 

.0229 


0.0230 

.0231 

.0232 

.0233 

.0234 


0.0235 

.0236 

.0237 

.0238 

.0239 


0.0240 

.0241 

.0242 

.0243 

.0244 


0.0245 

.0246 

.0247 

.0248 

.0249 


€ 

JC 

e~ x 

.98020 

0.0250 

0.97531 

.98010 

.0251 

.97521 

.98000 

.0252 

.97511 

.97990 

.0253 

.97502 

.97981 

.0254 

.97492 

.97971 

0.0255 

0.97482 

.97961 

.0256 

.97472 

.97951 

.0257 

.97463 

.97941 

.0258 

.97453 

.97932 

.0259 

.97443 

1.97922 

0.0260 

0.97434 

.97912 

.0261 

.97424 

.97902 

.0262 

.97414 

.97893 

.0263 

.97404 

.97883 

.0264 

.97395 

).97873 

0.0265 

0.97385 

.97863 

.0266 

.97375 

.97853 

.0267 

.97365 

.97844 

.0268 

.97356 

.97834 

.0269 

.97346 

0.97824 

0.0270 

0.97336 

.97814 

.0271 

.97326 

.97804 

.0272 

.97317 

.97795 

.0273 

.97307 

.97785 

,0274 

.97297 

0.97775 

0.0275 

0.97287 

.97765 

.0276 

.97278 

.97756 

.0277 

.97268 

.97746 

.0278 

.97258 

.97736 

.0279 

.97249 

0.97726 

0.0280 

0.97239 

.97716 

.0281 

.97229 

.97707 

.0282 

.97219 

.97697 

.0283 

.97210 

.97687 

.0284 

.97200 

0.97677 

0.0285 

0.97190 

.97668 

.0286 

.97181 

.97658 

.0287 

.97171 

.97648 

.0288 

.97161 

.97638 

.0289 

,97151 

0.97629 

0.0290 

0.97142 

.97619 

.0291 

.97132 

.97609 

.0292 

.97122 

.97599 

.0293 

.97113 

.97590 

,0294 

.97103 

0.97580 

0.0295 

0.97093 

,97570 

.0296 

.97083 

.97560 

.0297 

.97074 

.97550 

.0298 

.97064 

.97541 

.0299 

.97054 
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TABLE A- 1 .—Continued. 


X 

e~ x 

X 


X 

e~ x 

X 

e~ x 

X 

e~ x 

X 

e’ x 

0.0300 

0.97045 

0.0350 

0.96561 

0.0400 

0.96079 

0,0450 

0.95600 

0.0500 

0.95123 

0,0550 

0.94649 

.0301 

.97035 

.0351 

.96551 

.0401 

.96069 

.0451 

.95590 

.0501 

.95113 

.0551 

.94639 

.0302 

.97025 

.0352 

.96541 

.0402 

.96060 

.0452 

.95581 

.0502 

.95104 

.0552 

.94630 

.0303 

.97015 

.0353 

.96531 

.0403 

.96050 

.0453 

.95571 

.0503 

.95094 

.0553 

.94620 

.0304 

.97006 

.0354 

.%522 

.0404 

.96041 

.0454 

.95562 

.0504 

.95085 

.0554 

.94611 

0.0305 

0.969% 

0.0355 

0.%512 

0.0405 

0.96031 

0.0455 

0.95552 

0.505 

0.95075 

0.0555 

0.94601 

.0306 

.96986 

.0356 

.%503 

.0406 

.96021 

.0456 

.95542 

.0506 

.95066 

.0556 

.94592 

.0307 

.96977 

.0357 

.96493 

.0407 

.96012 

.0457 

.95533 

.0507 

.95056 

.0557 

.94582 

.0308 

.96967 

.0358 

.96483 

.0408 

.96002 

.0458 

.95523 

.0508 

.95047 

.0558 

.94573 

.0309 

.96957 

.0359 

.96474 

.0409 

.95993 

.0459 

.95514 

.0509 

.95037 

.0559 

.94563 

0.0310 

0.96948 

0.0360 

0.96464 

0.0410 

0.95983 

0.0460 

0.95504 

0.0510 

0.95028 

0.0560 

0.94554 

.0311 

.96938 

.0361 

.96454 

.0411 

.95973 

.0461 

.95495 

.0511 

.95018 

.0561 

.94544 

.0312 

.96928 

.0362 

.96445 

.0412 

.95964 

.0462 

.95485 

.0512 

.95009 

.0562 

.94535 

.0313 

.96918 

.0363 

.96435 

.0413 

.95954 

.0463 

.95476 

.0513 

.94999 

.0563 

.94526 

.0314 

.96909 

.0364 

.96425 

.0414 

.95945 

.0464 

.95466 

.0514 

.94990 

.0564 

.94516 

0.0315 

0.96899 

0.0365 

0.96416 

0.0415 

0.95935 

0.0465 

0.95456 

0.0515 

0.94980 

0.0565 

0.94507 

.0316 

.96889 

.0366 

.96406 

.0416 

.95925 

.0466 

.95447 

.0516 

.94971 

.0566 

.94488 

.0317 

.96879 

.0367 

.%397 

.0417 

.94916 

.0467 

.95437 

.0517 

.94% 1 

.0567 

.94488 

,0318 

.%870 

.0368 

.%387 

.0418 

.95906 

.0468 

.95428 

.0518 

.94952 

.0568 

.94478 

,0319 

.96860 

.0369 

,%377 

.0419 

.95897 

.0469 

.95418 

.0519 

.94942 

.0569 

.94469 

0.0320 

0,96851 

0.0370 

0.%368 

0.0420 

0.95887 

0.0470 

0.95409 

0.0520 

0.94933 

0.0570 

0,94450 

.0321 

.96841 

.0371 

.96358 

.0421 

.95877 

.0471 

.95399 

.0521 

.94923 

.0571 

.94450 

.0322 

.96831 

.0372 

.%348 

.0422 

.95868 

.0472 

.95390 

.0522 

.94914 

.0572 

.94441 

.0323 

.96822 

.0373 

.%339 

.0423 

.95858 

.0473 

.95380 

.0523 

,94904 

.0573 

.94431 

.0324 

.96812 

.0374 

.96329 

.0424 

.95849 

.0474 

.95371 

.0524 

.94895 

.0574 

.94422 

0,0325 

0.96802 

0.0375 

0.963 19 

0.0425 

0.95839 

0.0475 

0.9536! 

0.0525 

0.94885 

0.0575 

0.94412 

.0326 

.%793 

.0376 

,%310 

.0426 

.95829 

.0476 

.95352 

.0526 

.94876 

.0576 

.94403 

.0327 

.96783 

.0377 

.96300 

.0427 

.95820 

.0477 

.95342 

.0527 

.94866 

.0577 

.94393 

.0328 

.96773 

.0378 

.96291 

.0428 

.95810 

.0478 

.95332 

.0528 

.94857 

.0578 

.94354 

.0329 

.%764 

.0379 

.96281 

.0429 

.95801 

XV79 

.95323 

.0529 

.94847 

.0579 

.94374 

0,0330 

0.%754 

0.0380 

0.96271 

0.0430 

0.95791 

0.0480 

0.95313 

0.0530 

0.94838 

0,0580 

0.94365 

,0331 

,%744 

.0381 

.96262 

,0431 

.94782 

.0481 

.95304 

.0531 

.94829 

.0581 

.94356 

.0332 

■%735 

.0382 

.%252 

,0432 

.95772 

.0482 

.95294 

.0532 

.94819 

.0582 

.94346 

.0333 

.%725 

.0383 

.%242 

.0433 

.95762 

.0483 

.95285 

.0533 

.94810 

.0583 

.94337 

.0334 

.%715 

.0384 

.96233 

.0434 

95753 

.0484 

.95275 

.0534 

.94800 

.0584 

.94327 

0.0335 

0.%705 

0.0385 

0.%223 

0.0435 

0.95743 

0.0485 

0,95266 

0.0535 

0.94791 

0.0585 

0.94318 

.0336 

.966% 

.0386 ; 

.96214 

.0436 

.95734 

.0486 

.95256 

.0536 

.94781 

.0586 

.94308 

.0337 

.96686 

.0387 

.%204 

.0437 

.95724 

.0487 

,95247 

.0537 

.94772 

.0587 

.94299 

.0338 

.96676 

.0388 

.96194 

.0438 

.95715 

.0488 

.95237 

.0538 

.94762 

.0588 

.94289 

.0339 

.96667 

.0389 

.96185 

.0439 

.95705 

.0489 

.95228 

.0539 

.94753 

.0589 

.94280 

0.0340 

0.96657 

0.0390 

0.96175 

0.0440 

0.95695 

0.0490 

0.95218 

0.0540 

0.94743 

0.0590 

0.94271 

.0341 

.96647 

.0391 

.96165 

.0441 

.95686 

.0491 

95209 

.0541 

94734 

.0591 

.94261 

.0342 

.96638 

.0392 

.96156 

.0442 

.95676 

.0492 

.95199 

.0542 

.94724 

.0592 

.94252 

.0343 

.96628 

.0393 

.%146 

.0443 

.95667 

.0493 

.95190 

.0543 

.94715 

.0593 

.94242 

.0344 

.96618 

.0394 

.96137 

.0444 

.95657 

.0494 

.95180 

.0544 

.94705 

.0594 

.94233 

0.0345 

0.96609 

0.0395 

0.96127 

0.0445 

0.95648 

0.0495 

0.95171 

0.0545 

0.946% 

0.0595 

0.94224 

.0346 

.96599 

.03% 

.96117 

.0446 

.95638 

.04% 

.95161 

.0546 

.94686 

.05% 

.94214 

.0347 

.96590 

.0397 

.96108 

.0447 

.95628 

.0497 

.95151 

.0547 

.94677 

.0597 

.94205 

.0348 

.96580 

.0398 

.96098 

.0448 

.95619 

.0498 

.95142 

.0548 

.94667 

.0598 

.94195 

.0349 

,%570 

.0399 

.96089 

.0449 

.95609 

.0499 

.95132 

.0549 

.94658 

.0599 

.94186 
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TABLE A- 1. —Continued. 


0.0600 

.0601 

.0602 

.0603 

.0604 

0.0605 

.0606 

.0607 

.0608 

.0609 

0.0610 

.0611 

.0612 

.0613 

.0614 

0.0615 

.0616 

.0617 

.0618 

.0619 

0.0620 

.0621 

.0622 

.0623 

.0624 

0.0625 

.0626 

.0627 

.0628 

.0629 

0.0630 

.0631 

.0632 

.0633 

.0634 

0.0635 

.0636 

.0637 

.0638 

.0039 

0.0640 

.0641 

.0642 

.0643 

.0644 

0.0645 

.0646 

.0647 

.0648 

.0649 


0.94176 

.94167 

.94158 

.94148 

.94139 

0.94129 

.94120 

.94111 

.941 C 1 

.94092 

0.94082 

.94073 

.94064 

.94054 

.94045 

0.94035 

.94026 

.94016 

.94007 

.93998 

0.93988 

.93979 

.93969 

.93960 

.93951 

0.93941 

.93932 

.93923 

.93913 

.93904 

0.93894 

.93885 

.93876 

.93866 

.93857 

0.93847 

.93838 

.93829 

.93819 

.93810 

0.93800 

.93791 

.93782 

.93772 

.93763 

0.93754 

.93744 

.93735 

.93725 

.93716 


0.0650 

.0651 

.0652 

.0563 

.0654 

0.0655 

.0656 

.0657 

.0658 

.0659 

0.0660 

.0661 

.0662 

.0663 

.0664 

0.0665 

.0666 

.0667 

.0668 

.0669 

0.0670 

.0671 

.0672 

.0673 

.0674 

0.0675 

.0676 

.0677 

.0678 

.0679 

0.0680 

.0681 

.0682 

.0683 

.0684 

0.0685 

.0686 

.0687 

.0688 

.0689 

0.0690 

.0691 

.0692 

.0693 

.0694 

0.0695 

.0696 

.0697 

.0698 

.0699 


0.93707 

.93697 

.93688 

.93679 

.93669 

0.93660 

.93651 

.93641 

.93632 

.93622 

0.93613 

.93604 

.93594 

.93585 

.93576 

0.93566 

.93557 

.93548 

.93538 

.93529 

0.93520 

.93510 

.93501 

.93491 

.93482 

0.93473 

.93463 

.93454 

.93445 

.93435 

0.93425 

.93417 

.93407 

.93398 

.93389 

0.93379 

.93370 

.93361 

.93351 

.93342 

0.93333 

.93323 

.93314 

.93305 

.93295 

0.93286 

.93277 

.93267 

.93258 

.93249 


0.0700 

.0701 

.0702 

.0703 

.0704 

0.0705 

.0706 

.0707 

.0708 

.0709 

0.0710 

.0711 

.0712 

.0713 

.0714 

0.0715 

.0716 

.0717 

.0718 

.0719 

0.0720 

.0721 

.0722 

.0723 

.0724 

0 0725 
.0726 
.0727 
.0728 
.0729 

0.0730 

.0731 

.0732 

.0733 

.0734 

0.0735 

.0736 

.0737 

.0738 

.0739 

0.0740 

.0741 

.0742 

.0743 

.0744 

0.0745 

.0746 

.0747 

.0748 

.0749 


0.93239 

.93230 

.93221 

.93211 

.93202 

0.93193 

.93183 

.93174 

.93165 

.93156 

0.93146 

.93137 

.93128 

.93118 

.93109 

0.93100 

.93090 

.93081 

.93072 

.93062 

0.93053 

.93044 

.93034 

.93025 

.93016 

0.93007 

.92997 

.92988 

.92979 

.92969 

0.92960 

.92951 

.92941 

.92932 

.92923 

0.92914 

.92904 

.92895 

.92886 

.92876 

0.92867 

.92858 

.92849 

.92839 

.92830 

0.92921 

.92811 

.92802 

.92793 

.92784 


0.0750 

.0751 

.0752 

.0753 

.0754 

0.0755 

.0756 

.0757 

.0758 

.0759 

0.0760 

.0761 

.0762 

.0763 

.0764 

0.0765 

.0766 

.0767 

.0768 

.0769 

0.0770 

.0771 

.0772 

.0773 

.0774 

0.0775 

.0776 

.0777 

.0778 

.0779 

0.0780 

.0781 

.0782 

.0783 

.0784 

0.0785 

.0786 

,0787 

.0788 

.0789 

0.0790 

.0791 

.0792 

.0793 

.0794 

0.0795 

.0796 

.0797 

.0798 

.0799 


0.92774 

.92765 

.92756 

.92747 

.92737 

0.92728 

.92719 

.92709 

.92700 

.92691 

0.92682 

.92672 

.92663 

.92654 

.92645 

0.92635 

.92626 

.92617 

.92608 

.92598 

0.92589 

.92580 

.92570 

.92561 

.92552 

0.92543 

.92533 

.92524 

.92515 

.92506 

0.92496 

.92487 

.92478 

.92469 

.92459 

0.92450 

.92441 

.92432 

,92422 

.92413 

0.92404 

.92395 

.92386 

.92376 

.92367 

0.92358 

.92349 

.92339 

.92330 

.92321 


0.0800 

.0801 

.0802 

.0803 

.0804 

0.0805 

.0806 

.0807 


.0809 

0.0810 

.0811 

.0812 

.0813 

.0814 

0.0815 

.0816 

.0817 

.0818 

.0819 

0.0820 

.0821 

.0822 

.0823 

.0824 

0.0825 

.0826 

.0827 

.0828 

.0829 

0.0830 

.0831 

.0832 

.0833 

.0834 

0.0835 

.0836 

.0837 

.0838 

.0839 

0.0840 

.0841 

.0842 

.0843 

.0844 

0.0845 

.0846 

.0847 

.0848 

.0849 


0.92312 

.92302 

.92293 

.92284 

.92275 

0.92265 

.92256 

.92247 

.92238 

.92229 

0.92219 

.92210 

.92201 

.92191 

.92182 

0.92173 

.92164 

.92155 

.92146 

.92136 

0.92127 

.92118 

.92109 

.92100 

.92090 

0.92081 

.92072 

.92063 

.92054 

.92044 

0.92035 

.92026 

.92019 

.92008 

.91998 

0.91989 

.91980 

.91971 

.91962 

.91952 

0.91943 

.91934 

.91925 

.91916 

.91906 

0.91897 

.91888 

.91879 

.91870 

.91860 


0.0850 

.0851 

.0852 

.0853 

.0854 

0.0855 

.0856 

.0857 

.0858 

.0859 

0.0860 

.0861 

.0862 

.0863 

.0864 

0.0865 

.0866 

.0867 

.0868 

.0869 

0.0870 

.0871 

.0872 

.0873 

.0874 

0.0875 

.0876 

.0877 

.0878 

.0879 

0.0880 

.0881 

.0882 

.0883 

.0884 

0.0885 


.0887 

.0888 

.0889 

0.0890 

.0891 

.0892 

.0893 

.0894 

0.0895 

.0896 

.0897 


.0899 


0.91851 

91842 

.91833 

,91824 

.91814 

0.91805 

.91796 

.91787 

.91778 

.91769 

0.91759 

.91750 

.91741 

.91732 

.91723 

0.91714 

.91704 

.91695 

.91686 

.91677 

0.91668 

.91659 

.91649 

.91640 

.91631 

0.91622 

.91613 

.91604 

.91594 

.91585 

0.91576 

.91567 

.91558 

.91549 

.91539 

0.91530 

.91521 

.91512 

.91503 

.91494 

0.91485 

.91475 

.91466 

.91457 

.91448 

0.91439 

.91430 

.91421 

.91411 

.91402 
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0.0900 0.91393 0.0950 0.90937 

.0901 .91384 .0951 .90928 

.0902 .91375 .0952 .90919 

.0903 .91366 .0953 .90910 

.0904 .91357 .0954 .90901 

0.0905 0.91347 0.0955 0.90892 

.0906 .91338 .0956 .90883 

.0907 .91329 .0957 .90874 

.0908 .91320 .0958 .90865 

.0909 .91311 .0959 .90855 

0.0910 0.91302 0.0960 0.90846 

.0911 .91293 .0961 .90837 

.0912 .91284 .0962 .90828 

.0913 .91274 .0963 .90819 

.0914 .91265 .0964 .90810 

0.0915 0.91256 0.0965 0.90801 

.0916 .91247 .0966 .90792 

.0917 .91238 .0967 .90783 

.0918 .91229 .0968 .90774 

.0919 .9122C .0969 .90765 

0.0920 0.92111 0.0970 0.90756 

.0921 .91201 .0971 .90747 

.0922 .91192 .0972 .90737 

.0923 .91183 .0973 .90728 

.0924 .91174 .0974 .90719 

0.0925 0.91165 0.0975 0.90710 

.0926 .91156 .0976 .90701 

.0927 .91147 .0977 .90692 

.0928 .91138 .0978 .90683 

.0929 .91128 .0979 .90674 


0.91119 

.91110 

.91101 

.9109- 

.91083 

0.91074 

.91065 

.91056 

.91046 

.91037 

0.91028 
.91019 
.91010 
.91001 I 
.90992 

0.90983 

.90974 

.90965 

.90955 

.90946 


0.0980 0.90665 

.0981 .90656 

.0982 .90647 

.0983 .90638 

.0984 .90629 

0.0985 0.90620 

.0986 .90611 

.0987 .90601 

.0988 .90592 

.0989 .90583 

0.0990 0.90574 
.0991 .90565 

.0992 .90556 

.0993 .90547 

.0994 .90538 

0.0995 0.90529 

.0996 .90520 

.0997 .90501 

.0998 .90502 

.0999 .90493 


0.90484 l 
.90475 
.90466 
.90457 
.90448 

0.90439 

.90429 

.90420 

.90411 

.90402 

0.90393 

.90384 

.90375 

.90366 

.90357 

0.90348 

.90339 

.90330 

.90321 

.90312 

0.90303 

.90294 

.90285 

.90276 

.90267 

0.90258 

.90249 

.90240 

.90231 

.90222 

0.90213 

.90204 

.90195 

.90186 

.90177 

0.90168 
.90159 
.90150 
.90141 
.90132 | 

i 0 90123 
.90114 
.90105 
I .90095 

l .90086 


0.90032 l 
.90023 
.90014 
.90005 
.89996 

0.89987 

.89978 

.89969 

.89960 

.89951 

0.89942 

.89933 

.89924 

.89915 

.89906 

0.89898 

.89889 

.89880 

.89871 

.89862 

0.89853 

.89844 

.89835 

.89826 

.89817 

0.89808 

.89799 

.89790 

.89781 

.89772 

0.89763 

.89754 

.89745 

.89736 

.89727 

0.89718 

.89709 

.89700 

.89691 

.89682 

i 0.89673 
.89664 
.89655 
.89646 
l .89637 


0.89583 

.89574 

.89565 

.89557 

.89548 

0.89539 

.89530 

.89521 

.89512 

.89503 

0.89494 

.89485 

.89476 

.89467 

.89458 

0.89449 

.89440 

.89431 

.89422 

.89413 

0.89404 
.89395 J 
.89387 
.89378 
.89369 

0.89360 

.89351 

.89342 

.89333 

.89324 


0.1150 0.89137 
.1151 .89128 

.1152 89119 

.1153 .89110 

.1154 .89101 

0.1155 0.89092 
.1156 .89083 

.1157 .89074 

.1158 .89063 

.1159 .89054 

0.1160 0.89041 
.1161 .89031 

.1162 .89031 

.1163 .89021 

.1164 .890C 

0.1165 0.89001 
.1166 .8899 

.1167 .8898: 

.1168 .8897i 

.1169 .8896 

0.1170 0.8895 
.1171 .8895 

.1172 .8894 

.1173 .8893 

.1174 .8892 

0.1175 0.8891 
.1176 889C 

.1177 .888$ 

.1178 .8881 

.1179 .8881 


0.89315 0.1180 0.888* 

.89306 .1181 .8881 

.89297 .1182 .888! 

.89288 .1183 .888* 

.89279 .1184 .888: 


0.89270 

.89261 

.89253 

.89244 

.89235 

0.89226 

.89217 

.89208 


0.1185 0.888: 
.1186 .888 
.1187 .8881 

.1188 .887 

.1189 .887 

0.1190 0.987 
.1191 .88772 

.1192 .88763 


0.90077 3.1095 0.89628 

.90068 .1096 .89619 

.90059 .1097 .89610 

.90050 .1098 .89601 

.90041 .1099 .89592 


.89199 .1193 

,w 

.89190 .1194 

.887 

1.89181 0.1195 

0.887 

.89172 .1196 

.887 

.89163 .1197 

.887 

.89154 .1198 

.887 

.89146 .1199 

.887 
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TABLE A- L— Continued. 


Jf 


jf 

r-.r 

X 

e -x 

X 

e ~* 

X 

e-* 

X 


0.1200 

0.88692 

0.1250 

0.88250 

0.1300 

0.87810 

0.1350 

0.87372 

0.1400 

0.86936 

0.1450 

0.86502 

.1201 

.88683 

.1251 

.88241 

.13 i 

.87801 

.1351 

.87363 

.1401 

.wni 

.1451 

.86494 

.1202 

.88674 

.1252 

.88232 

.1302 

.87792 

.1352 

.87354 

.1402 

.86918 

.1452 

.86485 

.1203 

.88665 

.1253 

.88223 

.1303 

.87783 

1353 

.87345 

.1403 

.86910 

.1453 

.86476 

.1204 

.88657 

.1254 

.88214 

.1304 

.87774 

.1354 

.87337 

.1404 

.86901 

.1454 

.86468 

0.1205 

0.88648 

0.1255 

0.88206 

0.1305 

0.87766 

0.1355 

0 87328 

0.1405 

0.86892 

0.1455 

0.86459 

.1206 

.88639 

.1256 

.88197 

.1306 

.87757 

.1356 

.87319 

.1406 

.86884 

.1456 

.86450 

.1207 

.88630 

.1257 

.88188 

.1307 

.87748 

.1357 

.87310 

.1407 

.86875 

.1457 

.86442 

.1208 

.88621 

.1258 

.88179 

.1308 

.87739 

.1358 

.87302 

.1408 

.86866 

.1458 

.86433 

.1209 

.88612 

,1259 

.88170 

.1309 

.87731 

.1359 

.87283 

.1409 

.86858 

.1459 

.86424 

0.1210 

0.88603 

0.1260 

0.88161 

0.1310 

0.87722 

0.1360 

0.87284 

0.1410 

0.86849 

0.1460 

0.86416 

.1211 

.88595 

.1261 

.88153 

.1311 

.87713 

.1361 

.87276 

.1411 

.86840 

.1461 

.86407 

.1212 

.88586 

.1262 

.88144 

.1312 

.87704 

.1362 

.87267 

.1412 

.86832 

.1462 

.86398 

.1213 

.88577 

.1263 

.88135 

.1313 

.87695 

.1363 

.87258 

.1413 

.86823 

.1463 

.86390 

.1214 

.88568 

.1264 

.88126 

.1314 

.87687 

.1364 

.87249 

.1414 

.86814 

.1464 

.86381 

0.1215 

0.88559 

0.1265 

0.88117 

0.1315 

0.87678 

0.1365 

0.87241 

0.1415 

0.86806 

0.1465 

0.86373 

.1216 

.88550 

.1266 

.88109 

.1316 

87669 

.1366 

.87232 

.1416 

.86797 

.1466 

.86364 

.1217 

.88541 

.1267 

.88100 

.1317 

.87660 

.1367 

.87223 

.1417 

.86788 

.1467 

.86355 

.1218 

.88533 

.1268 

.88091 

.1318 

.87652 

.1368 

.87214 

.1418 

.86779 

.1468 

. 86347 

.1219 

.88524 

.1269 

.88082 

.1319 

.87643 

.1369 

.87206 

.1419 

.86771 

.1469 

.86338 

0.1220 

0.88515 

0.1270 

0.88065 

0.1320 

0.87634 

0.1370 

0.87197 

0 1420 

0.86762 

0.1470 

0.86329 

.1221 

.88506 

.1271 


1321 

.87625 

.1371 

.87188 

.1421 

.86753 

.1471 

86321 

.1222 

.88497 

.1272 

.88056 

.1322 

.87617 

.1372 

.87180 

.1422 

.86745 

.1472 

.86312 

.1223 

.88488 

.1273 

.88047 

.1323 

.87608 

.1373 

.87171 

.1423 

.86736 

.1473 

.86304 

.1224 

.88479 

.1274 

.88038 

.1324 

.87599 

.1374 

.87162 

.1424 

.86727 

.1474 

.86295 

0.1225 

0.88471 

0.1275 I 

0.88029 

0.1325 

0.87590 

0.1375 

0.87153 

0.1425 

0.86719 

0.1475 

0.86286 

.1226 

.88462 

.1276 

.88021 

.1326 

.87582 

.1376 

.87145 

.1426 

.86710 

.1476 

.86278 

.1227 

.88453 

.1277 

.88012 

.1327 

.87573 

.1377 

.87136 

.1427 

.86701 

.1477 

.86269 

.1228 

.88444 

.1728 

.88003 

.1328 

.8 7 564 

.1378 

.87127 

.1428 

.86693 

.1478 

.86260 

.1229 

.88435 

.1279 

.87994 

.1329 

.87555 

.1379 

.87119 

.1429 

.86684 

,1479 

.86252 

0.1230 

0.88426 

0.1280 

0.87985 

0.1330 

0.87547 

0.1380 

0.87110 

0.1430 

0.86675 

0.1480 

0.86243 

.1231 

.88418 

.1281 

.87977 

.1331 

.87538 

1381 

.87101 

.1431 

.86667 

.1481 

.86234 

.1232 

.88409 

.1282 

.87968 

.1332 

.87529 

.1382 

.87092 

.1432 

.86658 

.1482 

.86226 

.1233 

.88400 

.1283 

.87959 

.1333 

.87520 

.1383 

.87084 

.1433 

.86649 

.1483 

.86217 

.1234 

.88391 

.1284 

.87950 

.1334 

.87511 

.1384 

.87075 

.1434 

.86641 

.1484 

.86209 

0.1235 

0.88382 

0.1285 

0.87941 

0.1335 

0.87503 

0.1385 

0.87066 

0.1435 

086632 

0.1485 

0.86200 

.1236 

.88373 

.1286 

.87933 

.1336 

.87494 

.1386 

.87058 

.1436 

,86623 

.1486 

.86191 

.1237 

.88364 

.1287 

.87924 

.1337 

.87485 

.1387 

.87049 

.1437 

.86615 

.1487 

86183 

.1238 

.88356 

.1288 

.87915 

.1338 

.87477 

.1388 

.87040 

.1438 

.86606 

.1488 

.86174 

.1239 

,88347 

.1289 

87906 

.1339 

.87468 

.1389 

.87031 

.1439 

.86597 

1489 

.86166 

0.1240 

0.88338 

0.1290 

0.87897 

0.1340 

0.87459 

0.1390 

0.87023 

0.1440 

0.86539 

0.1490 

0.86157 

.1241 

.88329 

.1291 

.87889 

.1341 

.87450 

.1391 

.87014 

.1441 

.86530 

.1491 

.86*48 

.1242 

.88320 

.1292 

87880 

.1342 

.87442 

.1392 

.87005 

.1442 

.86571 

.1492 

.86140 

.1243 

.88311 

.1293 

.87871 

.1343 

.87433 

.1393 

.86997 

.1443 

.86563 

.1493 

.86131 

.1244 

.88303 

.1294 

.87862 

.1344 

.87424 

.1394 

.86988 

.1444 

.86554 

.1494 

.86122 

0.1245 

0.88294 

0.1295 

0,87853 

0.1345 

0.87415 

0.1395 

0.86979 

0.1445 

086545 

0.1495 

0.86114 

.1246 

.88285 

.1296 

.87845 

.1346 

.87407 

.1396 

.86971 

.1446 

.86537 

.1496 

.86105 

.1247 

.88276 

.1297 

.87836 

.1347 

.87398 

.1397 

.86962 

.1447 

.86528 

.1497 

.86097 

.1248 

.88267 

.1298 

.87827 

.1348 

.87389 

.1398 

.86953 

J 448 

.86520 

.1498 

.86088 

.1249 

.88256 

.1299 

.87818 

.1349 

.87380 

.1399 

.86945 

.1449 

.86511 

.1499 

.86079 
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TABLE A-l .-ComiMied. 


X 

e~ x 

X 

e~ l 

X 

e~ * 

X 


V 


X 


0.1500 

0.86071 

0.1550 

0.85642 

0.1600 

0.85214 

0.1650 

0.84739 

0 1700 

0.84366 

0.1750 

0.83946 

.150! 

.86062 

.1551 

.85633 

.160! 

.85206 

.1651 

.8478) 

.1701 

.84358 

.1751 

.83937 

.1502 

.86054 

.1552 

.85624 

.1602 

.85197 

.1652 

.84772 

.1702 

.84350 

.1752 

.83929 

.1503 

.86045 

.1553 

.85616 

.1603 

.85189 

.165? 

.#4764 

.1703 

.84341 

.1753 

.8392! 

.1504 

.86036 

.1554 

.85607 

.1604 

.85180 

.1654 

.84755 

.1704 

.84333 

.1754 

.83912 

0.1505 

0.86028 

0.1555 

0.85599 

0.1605 

0.85172 

0.1655 

0.84747 

0 1705 

0.84324 

0.1755 

0.83904 

.1506 

86019 

.1556 

.85590 

.1606 

.85163 

.1656 

.84739 

.1706 

.84316 

.1756 

.83895 

.1507 

.86010 

.1557 

.85582 

.1607 

.85155 

.1657 

.84730 

.1707 

.84307 

.1757 

.83887 

.1508 

.86002 

.1558 

.85573 

.1608 

.85146 

.1658 

.84722 

.1708 

.84299 

.1758 

.83879 

.1509 

.85993 

.1559 

.85564 

.1609 

.85138 

.1659 

.84713 

.1709 

.84296 

.1759 

.83870 

0.1510 

0.85985 

0.1560 

0.85556 

0.1610 

0.85129 

0.1660 

0.84705 

0.1710 

0.84282 

0.1760 

0.83862 

.1511 

.85976 

.1561 

.85547 

.1611 

.85121 

.1661 

.84696 

.1711 

.84274 

.1761 

.83853 

.1512 

.85968 

.1562 

.85539 

.1612 

.85112 

.1662 

.84688 

.1712 

.84265 

.1762 

.8384* 

.1513 

.85959 

.1563 

.85530 

.1613 

.85104 

.1663 

84679 

.1713 

.84257 

.1763 

.838,7 

.1514 

.85950 

.1564 

.85522 

.1614 

.85095 

.1664 

.84671 

.1714 

.84248 

.1764 

.83828 

0.1515 

0.85942 

0.1565 

0.85513 

0.1615 

0.85087 

0.1665 

0.84662 

0.1715 

0.84240 

0.1765 

0.83820 

.1516 

.8593.? 

.1566 

.85505 

.1616 

.85078 

.1666 

.84654 

.1716 

.84231 

.1766 

.83811 

.1517 

.85925 

.1567 

.85496 

.1617 

.85070 

.1667 

.84645 

.1717 

.84223 

.1767 

.83803 

.1518 

.85916 

.1568 

.85488 

.1618 

.85061 

.1668 

.84637 

.1718 

.84215 

.1768 

.83795 

.1519 

.85907 

.1569 

.85479 

.1619 

.85053 

.1669 

.84628 

.1719 

.84206 

.1769 

.83786 

0.1520 

0.85899 

0.1570 

0.85470 

0.1620 

0.85044 

0.1670 

0.84620 

0.1720 

0.84198 

0.1770 

0.83778 

.1521 

.85890 

.1571 

.85462 

.1621 

.85036 

.1671 

.84611 

.1721 

.84189 

.1771 

.83770 

.1522 

.85882 

.1572 

.85453 

.1622 

.85027 

.1672 

.84603 

.1722 

.84181 

.1772 

.83761 

.1523 

.85873 

.1573 

.85445 

.1623 

.85019 

.1673 

.84595 

.1723 

.84173 

.1773 

.83753 

.1524 

.85864 

.1574 

.85436 

.1624 

.85010 

.1674 

.84586 

.1724 

.84164 

.1774 

.83744 

0.1525 

0.85856 

0.1575 

0.85428 

0.1625 

0.85002 

0.1675 

0.84578 

0.1725 

0.84156 

0.1775 

0.83736 

.1526 

.85847 

.1576 

.85412 

.1626 

.84993 

.1676 

.84569 

.1726 

.84147 

.1776 

.83728 

.1527 

.85839 

.1577 

.8541! 

.1627 

.84985 

.1677 

.8456) 

.1727 

.84139 

.1777 

.83719 

.1528 

.85830 

.1578 

.85402 

.1628 

.84976 

.1678 

84552 

.1728 

.8413! 

.1778 

.83711 

.1529 

.85822 

.1579 

.85394 

.1629 

.84968 

.1679 

.84544 

| 

.1729 

.84122 

.1779 

.83703 

0,1530 

0 X 5813 

0.1580 

! 0.85385 

0.1630 

0.84959 

0.1680 

0.84535 

0.1730 

0.84114 

0.178 . 

0.83694 

.153! 

.85804 

.1581 

.85376 

.1631 

.84951 

.1681 

.84527 

.1731 

.84105 

.1781 

.83686 

,1532 

.85796 

.1382 

.85368 

.1632 

.84942 

.1682 

.845)8 

.1732 

.84097 

.1782 

.83678 

.1533 

.85787 

.1583 

.85359 

.1633 

.84934 

.1683 

.84510 

.1733 

.84089 

.1783 

.83669 

.1534 

.85779 

.1584 

.85351 

.1634 

.84925 

.1684 

.84502 

.1734 

.84080 

.1784 

.83661 

0.1535 

0.85770 

0.1585 

0.85342 

0.1635 

0.84917 

0.1685 

0.84493 

0.1735 

0.84072 

O . I 785 

0.83652 

.1536 

.85761 

.1586 

.85334 

.1636 

.84908 

.1686 

.84485 

.1736 

.84063 

.1786 

.83644 

.1537 

.85753 

.1587 

.85325 

.1637 

.84900 

.1687 

.84476 

.1737 

.84055 

.1787 

.83636 

.1538 

.85744 

.1588 

.85317 

.1638 

.8489! 

.1688 

.84468 

.1738 

.84046 

.1788 

.8362? 

.1539 

.83736 

.1589 

.85308 

.1639 

.84883 

.1689 

.84459 

.1739 

.84038 

.1789 

.83619 

0.1540 

0.85727 

0.1590 

0.85300 

0.1640 

0.84874 

0.1690 

0.84451 

0.1740 

0.84030 

0.1790 

0.83611 

.1541 

.85719 

.159! 

.85291 

.1641 

.84866 

.169! 

.84442 

.1741 

.84021 

.1791 

.83602 

.1542 

.83710 

.1592 

.85283 

.1642 

.84857 

.1692 

.84434 

.1742 

.84013 

.1792 

.83594 

.1543 

.85701 

.1593 

.85274 

1 .1643 

.84849 

.1693 

.84426 

.1743 

.84004 

.1793 

.85586 

.1544 

.85693 

.1594 

.85266 

.1644 

.84840 

.1694 

.84417 

.1744 

.83996 

.1794 

.83577 

0.1545 

0.85684 

0.1595 

0.85257 

0.1645 

0.84832 

0.1695 

0.84409 

0.1745 

0.83988 

0.1795 

0.83569 

.1546 

.85676 

1596 

.85248 

.!5 4 6 

.84823 

.1696 

.84400 

.1746 

.83979 

.1796 

.83560 

.1547 

.85667 

.1597 

.85240 

.1647 

84815 

.1697 

.84392 

.1747 

.83971 

.1797 

.83552 

.1548 

.85659 

.1598 

.85231 

.1648 

.84806 

.1698 

.84383 

.1748 

.83962 

.1798 

.83544 

.1549 

.85650 

.1599 

.85223 

.1649 

.84798 

.1699 

.84375 

.1749 

.83954 

.1799 

.83535 
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TABLE A- 1.— Concluded. 


x 


e -i 


x 


0.1900 

.1801 

.1802 

803 

.1804 

0.1805 

.1806 

.1807 

.1808 

.1809 

0.1810 

.1811 

.1812 

.1813 

.1814 

0 1815 
.1816 
.1817 
1818 
.1819 

0.1820 

.1821 

.1822 

.1823 

.1824 

0.1825 

.1826 

.1827 

.1828 

.1829 

0.1830 

.1831 

.1832 

.1833 

.1834 

0.1835 

.1836 

.1837 

.1838 

.1839 

0.1840 

.1841 

.1842 

.1843 

.1844 

0.1845 

.1846 

.1847 

.1848 

.1849 


0.83527 

.83519 

.83510 

.83502 

.83494 

0.83485 

.83477 

.83469 

.83460 

.83452 

0.83444 

.83435 

.83427 

.83419 

.83410 

0.83402 

.83393 

.83385 

.83377 

.83368 

0.83360 

.83352 

.8334? 

.83335 

.83327 

0.83318 

.83310 

.83302 

.83293 

.83285 

0.83277 

.83268 

.83260 

.83252 

.83244 

0.83235 
.83227 
.83219 
.83210 
. .83202 

0.83194 

.83185 

.83177 

.83169 

.83160 

0.83152 

.83144 

.83135 

.83127 

.83119 


0.1850 

.1851 

.1852 

.1853 

.1854 

0.1855 

.1856 

.1857 

.1858 

.1859 

0.1860 

.1861 

.1862 

.1863 

.1864 

0.1865 

.1866 

.1867 

.1868 

.1869 

0.1870 

.1871 

.1872 

.1873 

.1874 

0.1875 

.1876 

.1877 

.1878 

.1879 

0.1880 

.1881 

.1882 

.1883 

.1884 

0.1885 

.1836 

.1887 

.1888 

.1889 

0.1890 

.1891 

.1892 

.1893 

.1894 

0.1895 

.1896 

.1897 

.1898 

.1899 


e~* 


0.83110 

.83102 

.83094 

.83085 

.83077 

0.83069 

.83061 

.83052 

.83044 

.83036 

0.83027 

.83019 

.83017 

.83002 

.82994 

0.82986 

.82978 

.82969 

.82961 

.82953 

0.82944 

.82936 

.82928 

.82919 

.82911 

0.82903 

.82895 

.82886 

.82878 

.82870 

0.82861 

.82853 

.82845 

.82837 

.82828 

0.82820 

82812 

.82803 

.82795 

.82787 

0,82779 

.82770 

.82762 

.82754 

.82746 

0.82737 

.82729 

.82721 

.82712 

.82704 


x 


0.1900 

.1901 

.1902 

.1903 

.1904 

0.1905 

.1906 

.1907 

.1908 

.1909 

0.1910 

.1911 

.1912 

.1913 

*914 

0.1915 

.1916 

.1917 

.1918 

.1919 

0.1920 

.1921 

.1922 

.1923 

.1924 

0.1925 

.1926 

.1927 

.1928 

.1929 

0.1930 

.1931 

.1932 

.1933 

.1934 

0.1935 

.1936 

.1937 

.1938 

.1939 

0.1940 

.1941 

.1942 

.1943 

.1944 

0.1945 

.1946 

.1947 

.1948 

.1949 




0.82696 

.82688 

.82679 

.82671 

.82663 

0.82655 

.82646 

.82638 

.82630 

.82622 

0.82613 

.82605 

.82597 

.82588 

.82580 

0.82572 

.82564 

.82555 

.82547 

.82539 

0.82531 

.82522 

.82514 

.82506 

.82498 

0.82489 

.82481 

.82473 

.82465 

.82456 

0.82448 

.82440 

.82432 

.32423 

.82415 

0.82407 

.82399 

.83391 

.82382 

.82374 

0.82366 

.82358 

.82349 

.82341 

.82333 

0.82325 

.82316 

.82308 

.82300 

.82392 


0.1950 

.1951 

.1952 

.1953 

.1954 

0.1955 

.1956 

.1957 

.1958 

.1959 

0.1960 

.1961 

.1962 

.1963 

.1964 

0.1965 

.1966 

1967 

1968 
.1969 

0.1970 

.1971 

.1972 

.1973 

.1974 

0.1975 

.1976 

.1977 

.1978 

.1979 

0.1980 

.1981 

.1982 

.1983 

.1984 

0.1985 

.1986 

.1987 

.1988 

.1989 

0.1990 

.1991 

.1992 

.1993 

.1994 

0.1995 

.1996 

.1997 

.1998 

.1999 


0.82283 

.82275 

.82267 

.82259 

.82251 

0.82242 

.82234 

.82226 

.82218 

.82209 

0.82201 

.82193 

.82185 

.82177 

.82168 

0.82160 

.82152 

.82144 

.82135 

.82127 

0.82119 

.82111 

.82103 

.82094 

.82086 

0.82078 

.82070 

.82062 

.82053 

.82045 

0.82037 

.82029 

.82021 

.82012 

.82004 

0.81996 

.81988 

.81980 

.81971 

.81963 

0,81955 

.81947 

.81939 

.81930 

.81922 

0.81914 

.81906 

.81898 

.81889 

.81881 






. . 




mSa 




































Probability, 

P, 


Sample size , N 


mrm 


5 

6 

7 

8 

9 

10 

11 

- 5.0 

0 

- 2.6271 

■sa 

- 2.8843 

- 2.9789 

- 3.0590 

- 3.1327 

- 3.1958 

- 4.0 

mmm 

- 2.0487 

EMI 

- 2.2612 

- 2.3404 

- 2.4052 

- 2.4667 

- 2.5188 

- 3.0 

■ 

- 1.4523 

- 1.5466 

- 1.6226 

- 1.6880 

- 1.7376 

- 1.7878 

- 1.8294 

- 2.0 


.8028 

.8810 

.9415 

.9923 

- 1.0351 

- 1.0740 

- 1.1071 

- 1.0 

.1586 

.0434 


.1235 

.1762 

.2227 

.2579 

.2893 

-0 


1.6808 

1.3681 

1.1900 

1.0602 

.%17 

.8914 

.8320 

.1 

.5398 

1.9138 

1.5664 

1.3628 

1.2168 

1.1126 

1.0351 

.9703 

.2 

.5792 

2.1557 

1.7665 

1.5439 

1.3850 

1.2706 

1.1844 

1.1137 

.3 

.6179 

2.4041 

1.9747 

1.7328 

1.5608 

1.4352 

1.3389 

1.2617 

.4 

.6554 

2.6582 

2.1986 

1.9285 

1.7380 

1.6061 

1.4975 

1.4138 

.5 

.6914 

2.9406 

2.4294 

2.1304 

1.9206 

1.7775 

1.6602 

1.5697 

.6 

.7257 

3.2293 

2.6662 

2.3378 

2.1082 

1.9522 

1.8270 

1.7295 

.7 

.7580 

3.5232 

2.9083 

2.5500 

2.3002 

2.1309 

1.9977 

1.8927 

.8 

.7881 

3.8217 

3 . 155 ! 

2.7665 

2.4961 

2.3133 

2.1719 

2.0591 

.9 

.8159 

4.1244 

3.4059 

2.9869 

2.6956 

2.4989 

2.3493 

2.2285 

1.0 

.8413 

4.4425 

3.6604 

3.2107 

2.8988 

2.6875 

2.5295 | 

2.4005 

1.1 

.8643 

4.7756 

3.9183 

3.4375 

3.1115 

2.8846 

2.7118 

2.5745 

1.2 

.8849 

5.1124 

4.1791 

3.6672 

3.3269 

3.0842 

2.8962 

2.7506 

1.3 

.9031 

5.4524 

4.4467 

3.9006 

3.5445 

3.2860 

3.0827 

2.9285 

1.4 

.9192 

5.7952 

4.7243 

4.1431 

3.7582 

3.4851 

3.2713 

3.1083 

1.5 

.9331 

6.1405 

5.0042 

4.3877 

3.9736 

3.6855 

3.4616 

3.2897 

1.6 

.9452 

6.4881 

5.2861 

4.6340 

4.1908 

3.8874 

3.6533 

3.4724 

1.7 

.9554 

6.8377 

5.5698 

4.8820 

4.4094 

4.0907 

3.8463 

3.6563 

1.8 

.9640 

7.1891 

5,8550 

5.1279 

4.6311 

4.2953 


3.8412 

1.9 

.9712 

7.5422 

6.1417 

5.3723 

4.8570 

4.5010 

4.2353 

4.0269 

2.0 

.9772 

7.8966 

6.4295 

5.6180 

5.0840 

4.7077 

4.4310 

4.2135 

2.1 

.9821 

8.2524 

6.7186 

5.8647 

5.3119 

4.9153 

4.6277 

4.4008 

2.2 

.9860 

8.6094 

7.0086 

6.1125 

5.5406 

5.1238 

4.8251 

4.5889 

2.3 

.9892 

8.9675 

7.2996 

6.3612 

5.7701 

5.3330 

5.0232 

4.7776 

2.4 

.9918 

9.3265 

7.5914 

6.6107 

6.0003 

5.5429 

5.2219 

4.9670 

2.5 

.9937 

9.6865 

7.8849 

6.8609 

6.2311 

5.7534 

5.4212 

5.1568 

2.6 

.9953 

1.0472 

8.1772 

7.1119 

6.4624 

5.9646 

5.6210 

5.3472 

2.7 

.9965 

1.4011 

8.4694 

7.3635 

6.6943 

6.1762 

5.8213 

5,5380 

2.8 

.9974 

1.7549 

8.7588 

7.6191 

6.9248 

6.3894 

6.0221 

5.7292 

2.9 

.9981 

11.1094 

9.0488 

7,8753 

7.1549 

6.6040 

6.2232 

5.9207 

3.0 

.9986 

11.4647 

9.3395 

8.1319 

7.3855 

6.8191 

6.4247 

6.1126 

3.1 

.9990 

11.8207 

9.6307 

8.3889 

7.6165 

7.0345 

6.6266 

6.3047 

3.2 

.9993 

12.1773 

9.9223 

8.6463 

7.8479 

7.2502 

6.8287 

6.4972 

3.3 

.9995 

12.5345 

1.2145 

8.9040 

8 . 07 % 

7.4662 

7.0312 

6.6900 

3.4 

(WU 

.WO 

12.8922 

1.5070 

9.1620 

8.3117 

7.6825 

7.2339 

6.8830 

3.5 

9997 

13.2505 

1.8000 

9 . 42C3 

8.5440 

7.8990 

7.4368 

7 0762 

3.6 

.9998 

13.6092 

11.0933 

9.6739 

8.7767 

8.1157 

MEZ&Ml 

7.2697 

3.7 

.9998 

i 2 9684 

11.3870 

9.9377 


8.3326 

7,8435 

7.4633 


12 


- 3.2521 

- 2.5652 

- 1.8664 

- 1.1364 

.3168 

.7833 

.9175 

1.0563 

1.1994 

1.3463 

1.4970 

1.6512 

1.8085 

1.9689 

2.1320 

2.2975 

2.4650 

2.6344 

2.8056 

2.9785 

3.1528 

3.3284 

3 . 505 ! 

3.6828 

3.8613 

4.0405 

4.2205 

4.4012 

4.5825 

4.7644 

4.9468 

5.1296 

5.3129 

5.4965 

5.6804 

5.8647 

6.0492 

6.2340 

6.4191 

6.6044 

6.7899 

6.9756 

7.1616 




















Probability 

Pr 



TABLE A-3.— Continued, 
(a) Concluded. 


Sample size, N 


5 

6 

7 

8 

9 

10 

11 

12 

14.3280 

11.6809 

10.1968 

9.2427 

8.5497 

8.0471 

7.6572 

7.3477 

14.6880 

11.9752 

10.4560 

9.4761 

8.7671 

8.2500 

7.8512 

7.5340 

15.0488 

12.2698 

10.7155 

9.7097 

8.9845 

8.4548 

8.0454 

7.7204 

15.4090 

12.5646 

10.9751 

9.9435 

9.2022 

8.6590 

8.2397 

7.9069 

15.7700 

12.8597 

11.2350 

10.1775 

9.4200 

8.8632 

84342 

8.0937 

16.1313 

13.1550 

11.4950 

10.4116 

9.6379 

9.0677 

8.6288 

8.2805 

16.4929 

13.4505 

11.7551 

10.6459 

9.8559 

9.2722 

8.8235 

8.4674 

16.8547 

13.7463 

12.0154 

10.8804 

10.074! 

9.4769 

9.0184 

8.6545 

17.2168 

14.0422 

12.2758 

11.1150 

10.2924 

9.6817 

9.2134 

8.8417 

17.5792 

14.3383 

12.5364 

11.3497 

10.5108 

9.8866 

9.4084 

9.0289 

17.9417 

14.6346 

12.7970 

11.5846 

10.7293 

10.0917 

9.6036 

9.2163 

18.3045 

14.9310 

13.0578 

11.8196 

10.9479 

10.2968 

9.7989 

9.4038 

18.6674 

15.2276 

13.3187 

12.0547 

11.1666 

10.5020 

9.9942 

9.5913 

19.0306 

15.5243 

13.5797 

12.2900 

11.3854 

10.7073 

10.1896 

9.7789 

19.3939 

15.8212 

13.8408 

12.5253 

11.6043 

10.9127 

10.3851 

9.9666 

19.7574 

16.1182 

14.1020 

12.7607 

11.8232 

11.1181 

10.5807 

10.1543 

20.1210 

16.4153 

14.3632 

12.9962 

12.0422 

11.3237 

10.7764 

10.3422 

20.4848 

16.7125 

14.6246 

13.2318 

12.2613 

11.5293 

10.9721 

10.5300 

20.8488 

17.0099 

14.8860 

13.4675 

12.4804 

11.7350 

11,1679 

10.7180 

21.2129 

17.3073 

15.1475 

13.7033 

12.6996 

11.9407 

11.3638 

10.9060 

21.5771 

17.6049 

15.4091 

13.9391 

12.9189 

12.1465 

11.5597 

11.0941 

21.9414 

17.9025 

15.6707 

14.1751 

13.1382 

12.3524 

11.7556 

11.2822 

22.3059 

18.2003 

15.9324 

14.4110 

13.3576 

12.5583 

11.9516 

11.4703 

22.6705 

18.4981 

16.1941 

14.6471 

13.57/0 

12.7643 

12.1477 

11.6585 

23.0351 

18.7960 

16.4560 

14.8832 

13,7965 

12.9703 

I2.343G 

11.8468 

23.3999 

19.0940 

16.7178 

15 i 194 

14.0160 

13.1763 

12.5400 

12.0351 

23.7648 

19.3921 

16.9797 

15.3556 

14.2356 

13.3825 

12.7362 

12.2234 

24.1298 

19.6902 

17.2417 

15.5919 

14.4552 

13.5886 

12.9324 

12.4118 

24.4948 

19.9884 

17.5037 

IS.8282 

14.6748 

13.7948 

13.1287 

12.6002 

24.8600 

20.2867 

17.7658 

16.0646 

14.8945 

14.0011 

13.3250 

12.7887 

25.2252 

20.5850 

18.0279 

16.3011 

15.1143 

14.2074 

13.5214 

12.9771 

25.5905 

20.8834 

18.2901 

16.5375 

15.3340 

14.4137 

13.7177 

13.1657 

25.9559 

21.1819 

18.5523 

16.7741 

15.5538 

14.6200 

13.9142 

13.3542 

26.3214 

21.4804 

18.8145 

17.0106 

15.7736 

14.8264 

14.1106 

13.5428 

26.6869 

21.7789 

19.0768 

17.2472 

15.9935 

15.0328 

14.3071 

13.7314 

27.0525 

22,0776 

19.3391 

17.4839 

16.2134 

15.2393 

14.5036 

13.9200 

27.4182 

22.3762 

19.6014 

17.7206 

16.4333 

15.4458 

14.7002 

14.1087 

27.7839 

22.6749 

19.8638 

17.9573 

16.6533 

15.6523 

14.8967 

14.2974 

28.1497 

22.9737 

20.1262 

18.1940 

16.8732 

15.8588 

15.0933 

14.4861 

28,5155 

23.2725 

20.3886 

18.4308 

17.0932 

16.0654 

15.2900 

14.6748 

28.8814 

23,5714 

20.6511 

18.6676 

17,3133 

16.2720 

15.4866 

14.8636 

29.2474 

23.8702 

20.9136 

18.9045 

17.5333 

16.4786 

15.6833 

15.0524 

29,6134 

24.1692 

21.1761 

19.1414 

17.7534 

16.6852 

15.8800 

IS. 2412 
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TABLE A-3.— Continued, 
(b) Sample size* 13 to 20 


Sample size, N 


13 

14 

15 

-3.3027 

-3.3485 

-3.3903 

-2.6069 

-2.6447 

-2.6792 

-1.8997 

-1.9299 

-1.9573 

-1.1628 

-1.1866 

-1.2083 

-.3411 

-.3628 

-.3823 

.7424 

.7074 

.6770 

.8733 

.8357 

.8031 

1.0083 

.9679 

.9328 

1.1477 

1.1039 

1.0662 

1.2905 

1.2433 

1.2028 

1.4369 

1.3862 

1.3428 

1.5867 

1.5324 

1.4859 

1.7393 

1.6810 

1.6312 

1.8947 

1.8324 

1.7792 

2.0527 

1.9862 

1.9294 

2.2130 

2.1422 

2.0818 

2.3752 

2.3000 

2.2359 

2.5393 

2.4596 

2.3917 

2.7050 

2.6208 

2.5491 

2.8722 

2.7833 

2.7077 

3.0408 

2.9472 

2.8675 

3.2106 

3.1122 

3.0285 

3.3815 

3.2782 

3.1905 

3.5534 

3.4452 

3.3533 

3.7259 

3.6129 

3.5168 

3.8992 

3.7812 

3.6810 

4.0732 

3.9503 

3.8459 

4.2479 

4.1200 

4.0113 

4.4232 

4.2902 

4.1773 

4.5990 

4.4610 

4.3438 

4.7753 

4.6322 

4.5107 

4.9520 

4.8038 

4.6781 

5.1291 

4.9759 

4.8458 

5.3066 

5.1482 

5.0138 

5.4843 

5.3208 

5.1820 

5.6624 

5.4937 

5.3505 

5.8407 

5.6668 

5.5193 

6.0192 

5.8402 

5.6882 

6.1981 

6.0138 

5.8575 

6.3771 

6.1876 

6.0269 

6.5564 

6.3617 

6.1965 

6.7358 

6.5359 

6.3663 

6.9154 

6.7103 

6.5363 


-1.9826 


-2.0275 

-1.2281 

-1.24U 

-1.2633 

-.4000 

-.4162 

-4309 

.6503 

.6265 

.6049 

7745 

.7491 

.7260 


.8751 

.8503 

1.0332 

1.0042 

.9777 

1.1675 

1.1364 

1.1081 

1.3049 

1.2717 

1.2414 

1.4454 

1.4099 

1.3775 

1.5880 

1.5500 

1.5155 

1.7331 

1.6926 

1.6558 

1.8803 

1.8372 

1.7981 

2.0295 

1.9838 

1.9423 

2.1805 

2.1320 

2.0880 

2.3330 

2.2817 

2.2352 

2.4871 

2.4329 

2.3839 

2.6424 

2.5853 

2.5337 

2.7988 

2.7388 

2.6845 

2.9563 

2.8932 

2.8363 

3.1147 

3.0486 

2.9890 

3.2740 

3.2049 

3.1425 

3.4340 

3.3617 

3.2966 

3.5946 

3.5193 

3.4513 

3.7559 

3.6774 

3.6066 

* 3.9177 

3.8360 

3.7625 

4.0800 

3.9952 

3.9188 

4.2429 

4.1548 

4.0756 

4.4061 

4.3149 

4.2327 

4.5697 

4.4753 

4.3903 

4.7337 

4.6361 

4.5482 

4.8980 

4.7971 

4,7063 

5.0625 

4.9584 

4.8647 

5.2273 

5.1198 

5.0232 

5.3922 

5.2816 

5.1820 

5.5575 

5.4435 ! 

5.3411 

5.7229 

5.6056 

5.5003 

5.8885 

5.7680 

5.6597 

6.0544 

5.9305 

5.8193 

6.2204 

6.0932 

5.9790 

6.3865 

6.2560 

6.1389 





























^ O' 


Probability, 

P, 


Sample size, N 


18 


19 


20 




13 

14 

7.0952 

6.8849 

7.2752 

7.0596 

7.4553 

7.2344 

7.6356 

7.4094 

7.8159 

7.5845 

7.9964 

7.7597 

8.1771 

7.9351 

8.3578 

8. 1 105 

8.5386 

8.2861 

8.7195 

8.4617 

8.9005 

8.6374 

9.0816 

8.8132 

9.2628 

8.9890 

KZHH 

9.1650 

9.6253 

9.3410 

9.8067 

9.5170 

9.9881 

9.6932 

10.1696 

9.8694 

10.3512 

10,0456 

!Im9 

10.2219 


10.8962 
1 1.0779 

Im9 

11.2598 

10.9276 

11.4416 

11.1041 

11.6235 

M .2806 

11.8054 

11.4572 

11.9874 

11.6339 

12.1694 

11.8105 

12.3514 

11.9873 

■ •$£9 

12.1640 

12.3407 

12.8978 

12.5175 

ERE59 

12.6944 

12.8712 

13.4443 

13.0481 

13.6266 

13.2250 

13.8088 

13.4019 

13.9911 

13,5788 

14.1734 

13.7558 

KKgl 

IKE 79 

13.9328 

14.1098 

14.7205 

14.2868 


9.2715 

9.4431 

9.6148 

9.7865 

9.9583 

10.1302 

10.3020 

10.4740 

10.6459 

10.8179 

10.9900 

11.1621 

11.3342 

11.5063 

11.6785 

11.8507 

12.0230 

12.1952 

12.3675 

12.5398 

12.7122 

12.8846 

13.0569 

13.2294 

13.4018 

13.5742 

13.7467 

13.9192 


9.2280 

9.3958 

9.5637 

9.7316 

9.8996 

10.0676 

iU .2356 

10.4037 

10.5718 

10.7400 

10.9082 

11.0764 

11.2447 

11.4130 

11.5813 

11.7496 

11.9180 

12.0864 


12.9288 

13.0973 

13.2659 

13.4344 

13.6030 



6.2990 

6.1925 

6.0974 


6.4591 

6.3501 

6.2526 


6.6194 

6.5077 

6.4079 

6.9088 

6.7798 

6.6655 

6.5633 

7.0723 

6.9403 

6.8234 

6.7189 

7.2359 

7.1010 

6.9814 

6.8745 

7.3995 

7.2617 

7.1394 

7.0302 

7.5633 

7.4225 

7.2976 

7.1860 

7.7272 

7.5833 

7.4558 

7.3419 

7.8911 

7.7443 

7.6141 

7.4978 

8.0551 

7.9053 

7.7725 

7.6538 

8.2192 

8.3834 

8.0664 

8.2276 

7.9310 

8.0895 

■ESI 

8.5476 

8.3888 

8.2480 

8.1222 

8.7119 

8.5501 

8.4067 

8.2785 

8.8763 

8.7114 

8.5654 

8.4348 

9.0406 

8.8728 

8.7241 

8.5912 

9.2051 

9.0843 

8.8829 

8.7476 

9.3696 

9.1958 

9.0417 

8.9040 

9.5341 

9.3573 

9.2006 

9.0605 

9.6987 

9.5189 

9.3595 

9.2171 

9.8634 


9.5184 

9.3736 

10.0280 

9.8422 

9.6774 

9.5302 

10.1928 

10.0039 

9.8365 

9.6869 

10.3575 

10.1656 

9.9955 

9.8435 

10.5223 

10.3274 

10.1546 

10.0003 

10.6871 

10.4892 

10.3138 

10.1570 

10.8520 

10.6510 

10.4729 

10.3138 

11.0168 

10.8129 

10.6321 

10.4706 

11.1817 

10.9748 

10.7913 

10.6274 

11.3467 

11.1367 

10.9505 

10.7842 

11.5116 

11.2986 

11.1098 

10.9411 

11.6766 

11.4606 

11.2691 

11.0980 

11.8416 

11.6226 

11.4284 

11.2549 

12.0067 

11.7846 

11.5877 

11.4119 

12.1717 

11.9466 

11.7471 

11.5688 

12.3368 

12.1087 

11.9065 

11,7253 

12.5019 

12.2708 

12.0659 

11.8823 

12.6671 

12.4329 

12.2253 

12.0398 

12.8322 

12.5950 

12.3847 

12,1969 

12.9974 

12.7571 

12.5442 

12.3539 

13.1626 

12.9193 

12.7036 

12.5110 

13.3278 

13.0814 

12.8631 

12.6681 






























Safety 

margin. 

S M 

Probability. 

p. 


21 

22 

-5.0 

0 

-3.5836 

-3.6090 

-4.0 | 

0 

-2.8385 

-2.8594 

V 

.0013 

-2.0842 

-2.1008 

-2.0 

.0227 

-1.3075 

-1.3204 

-i.O 

.1586 

-.4688 I 

-.4797 

0 

.5000 

.5514 

.5366 

.1 

.5398 

.6691 

.6533 

.2 

.5792 

.7896 

.7728 

.3 

.6179 

.9130 

.8952 

.4 

.6554 

1.0390 

1.0201 

.5 

.6914 

1.1677 

1.1475 

.6 

.7257 

1.2988 

1.2773 

.7 

.7580 

1.4318 

1.4089 

.8 

.7881 

1.5669 

1.5426 

.9 

.8159 

1.7036 

1.6779 

1.0 

.8413 

1.8421 

1.8149 

l.i 

.8643 

1.9821 

1.9533 

1.2 

.8849 

2.1234 

2.0930 

1.3 

.9031 

2.2659 

2.2339 

1.4 

.9192 

2.4095 

2.3758 

1.5 

.9331 

2.5540 

2.5187 

1.6 

.9452 

2.6995 

2.6624 

1.7 

.9554 

2.8457 

2.8069 

1.8 

.9640 

2.9927 

2.9522 

1.9 

.9712 

3.1403 

3.0980 

2.0 

.9772 

3.2884 

3.2443 

2.1 

.9821 

3.4370 

3.3912 

2.2 

.9860 

3.5862 

3.5385 

2.3 

.9892 

3.7357 

3.6862 

2.4 

.9918 

3.8857 

3.8344 

2.5 

.9937 

4.0360 

3.9829 

2.6 

.9953 

4.1867 

4 1317 

2.7 

.9965 

4.3377 

4.2808 

2.8 

.9974 

4.4890 

4.4302 

2.9 

.9981 

4.6404 

4.5798 

3.0 

.9986 

4.7920 

4.7296 

3.1 

.9990 

4.9439 

4.8796 

3.2 

.9993 

5.0959 

5.0297 

3.3 

.9995 

5.2482 

5.1801 

3,4 

.9996 

5,4006 

5.3306 

3.5 

.9997 

5,5532 

5.4813 

3.6 

.9998 

5.7059 

5.6321 

3.7 

.9998 

5.8587 

5.7831 


Sample size. A' 


23 

24 

25 

26 

27 

28 

-3.6328 

-3.6554 

-3.6767 

-3.6970 

-3.7162 

-3.7346 



-2.9152 

-2.9318 

-2.9477 

-2.9628 

-2.1164 

-2.1312 

-2.1451 

-2.1584 

-2.1710 

-2.1830 

-1.3325 

-1.3439 

-1.3548 

-1.3650 

-1.3748 

-1.3840 

-.4900 

-.4996 

-.5087 

-.5173 

-.5254 

-.5331 

.5228 

.5101 

.4982 

.4872 

.4769 

.4671 

.6387 

.6253 

.6128 

.6011 

.5902 

5800 

.7574 

.7432 

.7299 

.7176 

.7061 

.6953 

.8788 

.8637 

.8496 

.8366 

.8244 

.8130 

1.0026 

.9866 

.9717 

.9579 

.9450 

.9330 

1.1289 

1.1119 

1.0961 

1.0814 

1.0678 

1.0550 

1.2576 

1.2395 

1.2227 

1.2071 

1.1926 

1.1791 

1.3880 

1.3687 

1.3510 

1.3345 

1.3191 

1.3048 

1.5204 

1.5000 

1,4811 

1.4637 

1.4474 

1.4323 

1.6544 

1 6327 

1.6128 

1.5943 

1.5771 

1.5611 

1.7900 

1.7671 

1.7460 

1.7265 

1.7084 

1.6914 

1.9270 

1.9028 

1.8806 

1.8600 

1.8408 

1.8230 

2.0652 

2.0397 

2.0163 

1.9945 

1.9744 

1.9556 

2.2046 

2.1778 

2.1531 

2.1302 

2.1090 

2.0892 

2.3451 

2.3169 

2.2909 

2.2669 

2.2446 

2.2238 

2.4864 

2.4568 

2.4296 

2.4044 

2.3810 

2.3592 

2.6286 

2.5976 

2.5690 

2.5426 

2.5182 

2.4954 

2.7716 

2.739! 

2.7093 

2.6817 

2.6561 

2.6322 

2.9152 

2.8813 

2.850! 

2.8213 

2.7946 

2.7697 

3.0594 

3.0241 

2.9915 

2.9615 

2.9336 

2.9077 

3.2041 

3.1673 

3.1334 

3.1021 

3.0731 

3.0461 

3.3493 

3.3110 

3.2758 

3.2432 

3.2130 

3.1849 

3.4950 

3.4552 

3.4186 

3.3847 

3.3534 

3.3242 

3.6411 

3.5998 

3.5618 

3.5267 

3.4941 

3.4638 

3.7876 

3.7448 

3.7053 

3.6689 

3.6352 

3.6038 

3.9344 

3.8901 

3.8492 

3.8115 

3,7766 

3.7441 

4.0816 

4.0357 

3.9934 

3.9544 

3.9183 

3.8847 

4.2290 

4.18(6 

4.1379 

4,0976 

4.0603 

4.0255 

4.3767 

4.3277 

4.2826 

4.2410 

4.2025 

4.1666 

4.5246 

4.4741 

4.4276 

4.3847 

4.3449 

4.3079 

4.6727 


4.5727 

4.5284 

4.4874 

4.4493 

4.8210 

4.7673 

4.7180 

4.6724 

4.6302 

4.5909 

4.9694 

4.9142 

4.8634 

4.8166 

4,7731 

4.7327 

5.1181 


5.0091 

4.9609 

4.9162 

4,8747 

5.2669 


5.1549 

5.1053 

5.0594 

5.0168 

5.4158 

5.3559 

5.3008 

5.2500 

5.2028 

5.1590 

5.5649 


5.4469 

5.3947 

5.3463 

5.3014 

5.7142 

5.6511 

5.5931 

5.5396 

5.4899 

5.4438 




























-.6406 
.3401 
.4472 
.5560 
.6665 
.7786 
.8922 
1.0073 
1.1236 
1.2411 
1.3596 
1.4792 
1.5996 
1.7208 
1.8429 
1.9656 
2.0888 
2.2126 
2.3369 
2.4617 
2.5868 
2.7123 
2.8380 
2.9641 
3.0905 
3.2171 
3.3439 
3.4709 
3.5982 | 
3.7256 | 
3.8531 
3.9808 
4.1086 
4.2366 
4.3646 
4.4928 
4.6211 
4.7494 
4.8778 


- 1.5536 

-.6692 

.3087 

.4146 

.5221 

.6311 

.7415 

.3533 

.9664 

1.0806 

1.1960 

1.3122 

1.4294 

1.5474 

1.6662 

1.7856 

1.9057 

2.0262 

2.1473 

2.2688 

2.3908 

2.5130 

2.6356 

2.7584 

2.8815 

3.0049 

3.1285 

3.2523 

3.3763 

3.5005 

3.6248 

3.7493 

3.8738 

3.9985 

4.1234 

4.2483 

4.3733 

4.4984 

4.6236 

4,7489 


- 4.1328 

- 3.2901 

- 2.4422 

- 1.5825 

-.6918 

.2846 

.3897 

.4963 

.6042 

.7134 

.8239 

.9356 

1.0483 

1.1621 

1.2767 

1.3922 

1.5084 

1.6253 

1.7429 

1.8610 

1.9795 

2.0986 

2.2180 

2.3379 

2.4580 

2.5784 

2.6991 

2.8201 

2.9412 

3.0626 

3.1842 

3.3059 

3.4278 

3.5499 

3.6720 

3.7943 

3.9167 

4.0393 

4.1619 

4.2845 

4.4073 

4.5302 

4.6531 


80 

90 

100 

- 4.1810 

- 4.2216 

- 4.2565 

- 3.3297 

- 3.3630 

- 3.3916 

- 2.4735 

- 2.4998 

- 2.5224 

- 1.6063 

- 1.6262 

- 1.6432 

- .7101 

-.7254 

-.7383 

.2655 

.2497 

.2364 

.3700 

.3537 

.3401 

.4758 

.4590 

.4449 

.5828 

.5654 

.5508 

.6912 

.6730 

.6579 

.8007 

.7818 

.7659 

.9113 

.8915 

.8750 

1.0229 

1.0022 

.9850 

1.1355 

1.1138 

1.0958 

1.2488 

1.2261 

1.2073 

1.3629 

1.3392 

1.3195 

1.4778 

1.4530 

1.4324 

1.5933 

1.5673 

1.5458 

1.7094 

1.6823 

1.6598 

1.8260 

1.7977 

1.7742 

1.9430 

1.9135 

1.8890 

2.0605 

2.0297 

2.0042 

2.1784 

2.1463 

2.1198 

2.2966 

2.2632 

2.2356 

2.4151 

2.3804 

2.3517 

2.5339 

2.4979 

2.4681 

2.6529 

2.6155 

2.5846 

2.7721 

2.7334 

2.7014 

2.8916 

2.8515 

2.8184 

3.0112 

2.9698 

2.9355 

3.1311 

3.0882 

3.0528 

3.2511 

3.2068 

3.1702 

3.3712 

3.3256 

3.2878 

3.4915 

3.4444 

3.4055 

3.6119 

3.5634 

3.5233 

3.7324 

3.6825 

3.6412 

3.8530 

3.8017 

3.7592 

3.9737 

3.9209 

3.8773 

4.0945 

4.0403 

3.9954 

4.2154 

4.1597 

4.1137 

4.3364 

4.2792 

4.2320 

4.4574 

4.3988 

4.3503 

4.5785 

4.5184 

4.4688 
























Probability 

P t 


TABLE A-3.— Concluded, 
(d) Concluded. 


Sample size, N 




5.5011 
5.6417 
5.7824 
5.9232 
6.0640 
6.2049 
6.3459 
6.4870 
6.6281 
6.7693 
6.9106 
7.0518 
7.1932 
7.3346 
7.4760 
7.6175 
7.7590 
7.9005 | 
8.0421 | 
8.1837 
8.3254 
8.4671 
8.6088 
8.7505 
8.8923 
9.0341 
9.1759 
9.3177 
9.4596 
9.6015 
9.7434 
9.8853 
10,0272 
10.1692 
10.3112 
10.4532 


m 


10.8792 

11.0213 

11.1633 

11.3054 

11.4475 


5.1969 
5.3301 
5.4634 
5.5968 
5.7302 
5.8637 
5.9972 
6.1308 
6.2645 
6.3982 
6.5319 
6.6657 
6.7996 
6.9334 I 
7.0673 
7.2013 
7.3353 
7.4693 
7.6033 
7.7374 
7.8715 
8.0056 
8.1398 
8.2740 
8.4082 
8.5424 
8.6766 
8.8109 
8.9452 
9.0794 
9,2138 
9.3481 
9.4824 
9.6168 
9.7512 
9.8856 
10,0200 
10.1544 
10.2888 
10.4233 
10.5577 
10 6922 
10.8266 


5.0064 
5.1350 J 
5.2636 
5.3923 
5.5211 
5.6500 
5.7789 
5.9078 
6.0368 
6.1659 
6.2949 
6.4241 
6.5532 I 
6.6824 
6.8116 
6.9409 
7.0702 
7.1995 
7.3288 
7.4582 
7 5876 
7.7170 
7. 84**4 
7.9759 
8.1054 
8.2348 
8.3644 
8.4939 
8.6234 
8.7530 
8.8826 
9.0122 
9.1418 
9.2714 
9.4010 
9.5307 
9.6603 
9.7900 
9.9197 
10.0494 
10.1791 
10.3088 
10.4385 



4.8742 

4.9996 

5.1251 

5.2506 

5.3762 

5.5019 

5.6275 

5.7533 

5.3791 

6.0049 

6.1307 

6.2566 

6.3825 

6.5085 

6.6345 

6.7605 

6.8865 

7.0126 

7.1386 

7.2648 

7.3909 

7.5170 

7.6432 

7.7694 

7.8956 

8.0218 

8.1481 

8.2744 

8.4006 

8.5269 

8.6532 

8.7795 

8.9059 

9.0322 

9.1586 

9.2849 

9.4113 

9.5377 

9.6641 

9.7905 

9.9169 

10.0433 

10.1698 


4.7761 
4.8991 
5.0223 
5.1454 
5.2686 
5.3919 
5.5152 
5.6385 
5.7619 
5.8853 
6.0088 
6.1323 
6.2558 
6.3794 
6.5029 
6,6266 
6.7502 I 
6.8738 
6.9975 
7.1212 
7.2449 
7.3687 
7.4924 
7.6162 
7.7400 
7.8638 
7.9876 
8.1114 
8.2353 
8.3592 
8.4830 
8.6069 
8.7308 
8.8547 
8.9786 
9.1026 
9.2265 
9.3505 
9.4744 
9.5984 
9.7224 
9.8464 


4.6997 

4.8209 

4.9422 

5.0635 

5.1849 

5.3063 

5.4777 

5.5492 

5.6708 

5.7923 

5.9139 

6.0356 

6.1572 

6.2789 

6.4006 

6.5224 

6.6441 

6.7659 

6.8877 

7.0095 

7.1314 

7.2532 

7.3751 

7.4970 

7.6189 

7.7409 

7.8627 

7.9847 

3.1067 

8.2286 

8.3506 

8.4726 

8.5946 

8.7167 

8.8387 

8.9607 

8.0828 

9.2048 

9.3269 

9.4490 

9.5711 

9.6932 

9.8153 


90 

100 

4.6381 

4.5872 

4.7579 

4.7058 

4.8777 

4.8243 

4.9975 

4.9430 

5.1174 

5.0615 

5.2373 

5.1803 

5.3573 

5.2991 

5.4773 

5.4178 

5.5974 

5.5367 

5.7174 

5.6555 

5.8375 

5.7744 

5.9577 

5.8933 

6.0778 

6.0122 

6.1980 

6.1311 

6.3182 

6.2501 

6.4384 

6.3691 

6.5587 

6.4881 

6.6790 

6.6071 

6.7993 

6.7262 

6.9196 

6.8452 

7.0399 

6.9643 

7.1603 

7.0834 

7.2806 

7.2025 

7.4010 

7.3217 

7.5214 

7.4408 

7.6418 

7.5600 

7.7622 

7.6791 

7.8827 

7.7983 

8.0G31 

7.9175 

8.1236 

8.0367 

8.2440 

8.1559 

8.3645 

8,2751 

8.4850 

8.3944 

8.6055 

8.5136 

8.7260 

8.6329 

8.8465 

8.7521 

8.9670 

8.8714 

9.0876 

8.9907 

9.2081 

9.1100 

9.3287 

9.2293 

9.4492 

9.3486 

9.5698 

9.4679 

9.6904 

9.5872 
















j>L 


Probability 

ft 


TABLE A-4.— SAFETY MARGINS AT 95-PERCENT CONFIDENCE LEVEL 
(a) Sample sires 5 to 12 

tv | Sample size, N 


-3.5814 

-3.6328 

-2.8364 

-2.8787 

-2.0819 

-2.1155 

-1.3044 

- 1 .3304 

-.4625 

-.4847 

.57% 

.5464 

.7017 

.6661 

.8273 

.7889 

.9562 

.9148 

1.0882 

1.0436 

1.2231 

1.1751 

1.3609 

1.3093 

1.5010 

1.4456 

1.6434 

1.5841 

1.7878 

1.7245 

1.9343 

1.8667 

2.0822 

2.0103 

2.2317 

2.1554 

2.3826 

2.3018 

2.5345 

2.4493 

2.6876 

2.5977 

2.8416 

2.7471 

2.9965 

2.8972 

3.1522 

3.0482 

3.3085 

3.1997 

3.4654 

3.3518 

3.6229 

3.5044 

3.7810 

3.6575 

3.9394 

3.8111 

4.0983 

3.9650 

4.2576 

4.1193 

4.4172 

4.2739 

4.577* 

4.4289 

4.7373 

4.5840 

4.8977 

4.7394 

5.0584 

4.8950 

5.2193 

5.0508 

5.3803 

5.2068 

5.5416 

5.3630 

5.7030 

5.5194 

5.8646 

5.6759 

6.0264 

5.8325 

6.1882 

5.9893 




m 





























TABLE A-4,*— Continued. 









-3.7191 

-3.7560 

—3.7895 

-3.8202 

-3.8485 

-3.8747 

-3.8990 

-2.9497 

-2.9800 


-3.0329 

-3.0561 


-3.0977 

-2.1719 

-2.1960 

-2.2179 

-2.2380 

-2.2564 

-2.2735 

-2.2894 

-1.3741 

-1.3927 

-1.4097 

-1.4251 

-1.4394 

-1.4525 

-1.4647 

-.5217 

-.5373 

-.5514 

-.5642 

-.5759 

-.5866 

-.5965 

.4943 

.4733 

.4547 

.4382 

.4234 

.4100 

.3978 

.6104 

.5881 

.5684 

.5510 

.5353 

.5212 

.5084 

.7293 

.7055 

.6846 

.6661 

.6496 

.6347 

.6211 

.8509 

.8256 

.8033 

.7837 

.7661 

.7503 

.7359 

.9751 


.9243 

.9034 

.8848 

.8679 

.8527 

1,1017 

1.0727 

1.0475 

1.0252 

1.0054 

.9875 

.9713 

1.2306 

1.1996 

1.1727 

1.1490 

1.1279 

1.1089 

1.0918 

1.3614 

1.3284 

1.2997 

1.2745 

1.2521 

1.2319 

1.2137 

1.4942 

1.4591 

1.4286 

1.4018 

1.3780 

1.3566 

1.3373 

1.6286 

1.5912 

1.5588 

1.5304 

1.5052 

1.4825 

1 4620 

1.7647 

1.7250 

1.6906 

2 6605 

1.6338 

1.6097 

1.5880 

1.9021 

1.8600 

1.8236 

1.79H 

1.7635 

1.7380 

1.7151 

2.0407 

1.9962 

1.9577 

1.9240 

1.8942 

1.8674 

1.8432 

2.1806 

2.1335 

2.0929 

2.0574 

2.0260 

1.9977 

1.9723 

2.3213 

2.2718 

2.2290 

2.1916 

2.1585 

2.1288 

2.1020 

2.4630 

2.41C9 

2.3659 

2.3265 

2.2918 

2.2606 

2.2325 

2.6055 

2.5507 

2.5035 

2.4622 

2.4258 

2.3930 

2.3636 

2.7487 

2.6913 

2.6418 

2.5986 

2.5605 

2.5261 

2.4954 

2.8926 

2.8325 

2.7807 

2.7355 

2.6957 

2.6598 

2.6277 

3.0370 

2.9743 

2.9202 

2.8730 

2.8313 

2.7930 

2.7604 

3.1820 

3.1165 

3.0601 

3.0108 

2.9674 

2 9284 

2.8935 

3.3274 

3.2592 

3.2004 

3.1491 

3.1040 

3.0633 

3.0270 

3.4733 

3.4023 

3.3411 

3.2878 

3.2409 

3.1986 

3.1608 

3.6196 

3.5458 

3.4823 

3.4269 

3.3781 

3.3343 

3.2950 

3.7662 

3.6896 

3.6237 

3.5662 

3.5156 

3.4702. 

3.4295 

3.9131 

3.8338 

3.7654 

3.7059 

3.6535 

3.6064 

3.5642 

4.0604 

3.9782 

3.9075 

3.8458 

3.7915 

3.7428 

3.6991 

4.2079 

4.1229 

4.0497 

3.9860 

3.9298 

3.8794 

3.8343 

4.3557 

4.2678 

4.1922 

4.1263 

4.0684 

4.0163 

3.9697 

4.5036 

4.4129 

4.3349 

4.2669 

4.2071 

4.1533 

4.1053 

4.6518 

4.5582 

4.4777 

4.4076 

4.3459 

4.2905 

4,2409 

4.8001 

4.7036 

4.6207 

4.5485 

4.4849 

4.4279 

4.3768 

4.9486 

4.8493 

4.7639 

4.6895 

4.6241 

4.5653 

4.5128 

5.0973 

4.9951 

4.9072 

4.8307 

4.7634 

4.7030 

4.6489 

5.2461 

5.1410 

5.0507 

4.9720 

4.9028 

4.8407 

4.7851 

5.3950 

5.2871 

5.1943 

5.1135 

5.4024 

4.9786 

4.9215 

5.5441 

5.4333 

5.3380 

5.2550 

5.1820 

5.1165 

5.0580 

5.6933 

5.5796 

5.4818 

5,3967 

5.3218 

5.2546 

5.1945 






























TABLE A-4.— Continued, 
(b) Concluded. 


Safety 

Probability 

margin, 

n 

S M 



San pie size, N 


5.8426 
5.9920 
6.1416 
6.2912 
6.4408 
6.5906 
6.7404 
6.8903 
7.0403 
7.1903 
7.3404 
7.4906 
7.6408 
7.7910 
7.9413 
8.0916 
8.2420 
8.3924 
8.5429 
8.6934 
8.8439 
8.9944 
9.1450 
9.2956 
9.4463 
9.5969 
9.7476 
9.8983 
10.0491 
10.1998 
10.3506 
10.5014 
10.6522 
10.8031 
10.9539 
11.1048 
11.2557 
11.4066 
11.5575 
11.7084 
11.83* 4 
12 . 01 * * 
12.1613 


5.7260 

5.8725 

6.0191 

6.1658 

6.3126 

6.4594 

6.6063 

6.7533 

6.9003 

7.0474 

7.1946 

7.3418 

7.4891 

7.6364 

7.7837 

7.9311 

8.0786 

8.2260 

8.3735 

8.5211 

8.6687 

8.8163 

8.9639 

9.1115 

9.2592 

9.4069 

9.5547 

9.7024 

9.8502 

9.9980 

10.1458 

10.2937 

10.4416 

10.5894 

10.7373 

10.8852 

11.0332 

11.1811 

11.3291 

11.4770 

11.6250 

11.7730 

11.9210 


5.6257 

5.7697 

5.9139 

6.0580 

6.2023 

6.3467 

6.4911 

6.6355 

6.7801 

6.9247 

7.0693 

7.2140 

7.3588 

7,5035 

7.6484 

7.7932 

7.9381 

8.0831 

8.2281 

8.3731 

8.5181 

8.6632 

8.8083 

8.9534 

9.0986 

9.2438 

9.3890 

9.5342 

9.6794 

9.8247 

9.9700 

10.1153 

10.2606 

10.4059 

10.5513 

10.6967 

10.8421 

10.9875 

11.1329 

11.2783 

11.4237 

11.5692 

11.7147 


5.5385 

5.6803 

5.8223 

5.9643 

6.1064 

6.2485 

6.3908 

6.5331 

6.6754 

6.8178 

6.9603 

7.1028 

7.2454 

7.3879 

7.5306 

7.6733 

7.8160 

7.9587 
8.1015 
8.2443 
8.3872 
8.5300 
8.6729 
8.8159 

8.9588 
9.1018 
9.2448 
9.3878 
9.5309 
9.6739 
9.8170 
9.9601 
10.1032 
10.2463 
10.3895 
10.5326 
10.67' 
10.81 
10.9622 
11.1054 
11.2487 
11.3919 
11.5352 


5.4617 

5.6016 

5.7417 

5.8818 

6.0220 

6.1622 

6.3025 

6.4429 

6.5834 

0.7239 

6.8644 

7.0050 

7.1456 

7.2863 

7.4270 

7.5677 

7.7085 

7.8494 

7.9902 

8.1311 

8.2720 

8.4129 

8.5539 

8.6949 

8.8359 

8.9770 

9.1180 

9.2591 

9.*C?7 

9.5413 

9.6825 

9.8236 

9.9648 

10.1060 

10.2472 

10.3884 

10.5296 

10.6709 

10.8122 

10.9534 

11.0947 

11.2360 

11.3773 


5.3928 

5.5310 

5.6694 

5.8078 

5.9463 

6.0848 

6.2234 

6.3621 

6.5008 

6.6396 

6.7784 

6.9173 

7.0562 

7.1951 

7.3341 

7.4731 

7.6.22 

7.7513 

7.8904 

8.0296 

8.1687 

8.3080 

8.4472 

8.5864 

8.7257 

8.8650 

9.0044 

9.1437 

9.2831 

94225 

9.5619 

9.7013 

9.8407 

9.9802 

10.1196 

10.2591 

10.3986 

10.5381 

10.6776 

10.8172 

10.9567 

11.0963 

11.2358 


5.3312 

5.4679 

5.6047 

5.7416 

5.8785 

6.0156 

6.1526 

6.2989 

6.4270 

6.5642 

6.7015 

6.8388 

6.9762 

7.1136 

7.2510 

7.3885 

7.5260 

7.6636 

7.8012 

7.9388 

8.0764 

8.2141 

8.3518 

8.4895 

8.6272 

8.7650 

8.9028 

9.0406 

9.1784 

9.3162 

9.4540 

9.5919 

9.7298 

9.8677 

10.0056 

10.1435 

10.2815 

10.4194 

10.5574 

10.6954 

10.8334 

10.9714 

11.1094 
































TABLE A-4,— Continued, 
(c) Concluded. 


Probability 

J> 

Sample size. N 

“t 

21 

22 

23 

24 

25 

26 

27 

28 

0.9999 

5.2254 

5.1795 

5.1375 

5.0988 

5.C631 

5.0300 

4.9992 

4.9704 



5.3595 

5.3125 

5 2695 

5.2298 

5.1933 

5.1593 

5.1278 

5.0983 



5.4937 

5.4456 

5.4015 

5.3609 

5.3235 

5.2887 

5.2564 

5.2262 



5.6280 

5.5787 

5.5336 

5.4921 

5.4538 

5.4182 

5.3851 

5.3542 



5.7623 

5.7119 

5.6658 

5.6233 

5.5841 

5.5477 

5.5139 

5.4823 



5.8967 

5.S-J52 

5.7980 

5.7546 

5.7145 

5.6773 

5.6427 

5.6104 



6.0311 

5.9785 

5.9303 

5.8859 

5.8449 

5.8069 

5.7716 

5.7386 



6.1657 

6.1119 

6.0626 

6.0173 

5.9754 

5.9366 

5.9005 

5.8668 



6.3002 

6.2453 

6.19id 

I 6.1487 

6.1060 

6.0663 

6.0295 

5.9951 



6.4348 

6.3788 

6.3274 

6.2802 

6.2366 

6.1961 

6.1585 

6.1234 



6.5695 

6.5123 

6.4599 

6.4117 

6.3672 

6.3259 

6.2875 

6.2517 

1.0000 

6.7042 

6.6458 

6.5924 

6.5433 

6.4979 

6.4558 

6.4166 

6.3801 



6.8389 

6.7794 

6.7250 

6.6748 

6.6286 

6.5856 

6.5457 

6.5085 



6.9737 

6.9131 

6.8575 

6.8065 

6.7593 

6.7156 

6.6749 

6.6369 



7.1085 

7.0467 

6.9902 

6.9381 

6.8901 

6.8455 

6.8041 

6.7654 



7.2433 

7.1804 

7.1228 

7.0698 

7.0209 

6.9755 

6.9333 

68939 



7.3782 

7.3141 

7.2555 

7.2015 

7.1517 

7.1055 

7.0625 

7.0224 



7.5131 

7.4479 

7.3882 

7.3333 

7.2826 

7.2355 

7.1918 

7.1510 



7.6480 

7.5817 

7.5209 

7.4651 

7.4134 

7.3656 

7.3211 

7.2795 



7.7830 

7.7155 

7.6537 

7.5969 

7.5443 

7.4957 

7.4504 

7.4081 



7.9180 

7.8493 

7.7865 

7.7287 

7.6753 

7.6258 

7.5797 

7.5368 



8.0530 

7.9832 

7.9193 

7.8605 

7.8062 

7.7559 

7.7091 

7.6654 



8.1880 

8.1171 

8.0521 

7.9924 

7.9372 

7.8861 

7 8385 

7.7941 



8.3231 

8.2510 

8.1850 

8.1243 

8.0682 

8.0162 

7.%79 

7.9228 



8.4582 

8.3849 

8.3179 

8.2562 

8.1992 

8.1464 

8.0973 

8.0515 



8.5933 

8.5189 

8.4508 

8.3881 

8.3303 

8.2766 

8.2267 

8.1802 



8.7284 

8.6529 

8.5837 

8.5201 

8.4613 

8.4069 

8.3562 

8.3089 



8.8635 

8.7868 

8.7166 

8.6520 

8.5924 

8.5371 

8.4857 

8.4377 



8.9987 

8.9208 

8.8496 

8.7840 

8.7235 

8.6674 

8.6152 

8.5664 



9.1338 

9.0549 

8.9825 

8.9160 

8.8546 

8.7976 

8.7447 

8.6952 



9.2690 

9.1889 

9.1155 

9.0480 

8.9857 

8.9279 

8.8742 

8.8240 



9.4042 

9.3230 

9.2485 

9.1801 

9.1168 

9.0582 

9.0037 

8.9528 



9.5395 

9.4570 

9.3815 

9.3121 

9.2480 

9,188j 

9.1333 

9.0817 



9.6747 

9.5911 

9.5146 

9.4442 

0.3791 

9.3189 

9.2628 

9.2105 



9.8099 

9.7252 

9.6476 

9.5762 

9.5103 

9.4492 

9.3924 

9.3394 



9.9452 

9.8593 

9.7807 

9.7083 

9.6415 

9.57% 

9.5220 

9.4682 



10.0805 

9.9934 

9.9137 

9.8404 

9.7727 

9.7099 

9.6516 

9.5971 



10.2158 

10.1276 

10.0468 

9.9725 

9.9039 

9.8403 

9.7812 

9.7260 



10.3511 

10.2617 

10.1799 

10.1046 

10.0351 

9,9707 

9.9108 

9.8549 



10.4864 

10.3959 

10.3130 

10.2368 

10.1664 

10.1011 

10.0404 

9.9838 



10.6217 

10.5300 

10.4461 

10.3689 

10.2976 

10.2315 

10.1700 

10.1127 



10.7570 

10.6642 

10.5792 

10.5010 

10.4288 

10.3619 

10.2997 

10.2416 



10.8924 

10.7984 

10.7123 

10,6332 

10.5601 

10.4923 

10.4293 

10.3705 














TABLE A-4.— Continued, 
(d) Sample sizes 30 to 100 


Probability, 


Sample size, N 





.0878 

-3.2528 

-2.4123 

-1.5589 

-.6717 

.3102 

.4168 

.5249 

.6347 

.7459 

.8586 

.9726 

1.0877 

1.2041 

1.3214 

1.4398 

1.5589 

1.6788 

1.7994 

1.9206 

2.0423 

2.1645 

2.2873 

2.4104 

2.5338 

2.6576 

2.7817 

2.9061 

3.0307 

3.1S55 

3.2905 

3.4058 

3.5311 

3.6567 

3.7824 

3.9082 

4.0341 

4.1601 

4.2863 

4.4125 

4.5388 

4,6652 

4.7917 


-4.1922 

-3.3386 

-2.4801 

-1.6105 

-.7122 

.2664 

.3713 

.4776 

.5852 

.6941 

.8042 

.9154 

1.0277 

1.1409 

1.2550 

1.3699 

1.4855 
1.6018 
1.7187 
1.8361 
1.9539 
2.0722 
2.1908 
2.3099 
2.4292 
2.5488 
2.6686 
2.7887 
2.9090 
3.0295 
3.1502 
3.2710 
3.3920 
3.5131 
3.6344 
3.7557 
3.8771 
3.9987 
4.1203 
4.2420 
4.3638 

4.4856 
4.6075 


>4.2661 

-3.3992 

-2.5280 

-1.6469 

-.7402 

.2371 

.3411 

.4462 

.5526 

.6600 

.7685 

.8781 

.9885 

1.0998 

1.2118 

1.3246 

1.4381 

1.5520 

1.6666 

1.7816 

1.8970 

2.0127 

2.1289 

2.2453 

2.3620 

2.4790 

2.5962 

2.7136 

2.8312 

2.9489 

3.0669 

3.1849 

3.3031 

3.4214 

3,5398 

3.6583 

3.7769 

3.8956 

4.0144 

4.1332 

4.2521 

4.3711 

4.4901 


•4.3220 

•3.4452 

-2.5642 

-1.6743 

-.7612 

.2158 

.3191 

.4235 

.5290 

.6354 

.7429 

.8512 

.9604 

1.0704 

1.1811 

1.2924 

1.4043 

1.5167 

1.6296 

1.7429 

1.8566 

1.9707 

2.0851 

2.1997 

2.3146 

2.4297 

2.5451 

2.6606 

2.7763 

2.8921 

3.0081 

3.1243 

3.2405 

3.3568 

3.4733 

3.5898 

3.7064 

3.8231 

3.9399 

4.0567 

4.1736 

4.2905 

4.4075 


-4.3664 
-3.4815 
-2.5929 
-1.6960 
-.7777 
.1993 
.3022 
.4060 
.5109 
.6166 
.7233 
.8308 
.9391 i 
1.0481 ! 
1.1577 
1.2680 
1.3787 
1.4900 
1.6017 
1.7138 
1.8262 
1.9390 
2.0521 
2.1654 
2.2789 
2.3927 
2.5066 
2.6207 
2.7350 
2.8494 
2.9640 
3.0787 ! 
3.1935 
3.3083 
3.4233 
3.5384 
3.6535 
3,7687 
3.8840 
3.9993 
4.1147 
4.2301 
4.3456 


-1.7138 | 
-.7911 
.1861 
.2886 
.3921 
.4965 
.6017 
.7078 
.8146 
.9222 
1.0304 
1.1393 
1.2487 
1.3586 
1.4689 
1.5797 
1.6908 
1.8023 I 
1.9140 | 
2.0261 ! 
2.1384 
2.2509 
2.3635 
2.4764 
2.5894 
2.7026 
2.8159 
2.9293 
3.0429 
3.1565 
3.2703 
3.3841 
3.4980 
3.6120 
3.7260 
3.8401 
3.9542 


-4.4333 i 
-3.5364 
-2.6361 
-1.7286 
-.8023 
.1752 
.2775 
.3806 
.4846 
.5894 
.6950 
.8014 
.9084 
1.0160 
1.1242 
1.2329 
1.3421 

1.4518 
1.5618 
1.6721 
1.7828 
1.8938 
2.0050 
2.1164 
2.2281 
2.3399 

2.4519 
2.5640 
2.6763 
2.7887 
2.9012 
3.0139 
3.1266 
3,2394 
3.3523 
3,4653 
3.5783 
3,6914 
3.8045 
3.9177 
4.0310 
4.1443 
4.2576 


-1.7413 

-.8118 

.1660 

.2681 

.3710 

.4747 

.5791 

.6843 

.7903 

.8968 

1.0039 

1.1116 

1.2198 

1.3284 

1.4374 

1.5468 

1.6566 

1.7666 

1.8769 

1.9874 

2.0981 

2.2091 

2.3202 

2.4314 

2.5428 

2.6544 

2.7661 

2.8778 

2.9897 

3.1017 

3.2137 

3.3258 

3.4380 

3.5503 

3.6626 

3.7750 

3.8874 

3.9998 

4.1123 

4.2249 
























































TABLE A-5.— SAFETY MARGINS AT 90-PERCENT CONFIDENCE LEVEL 
(a) Sample sizes 5 to 12 



-3.5162 

-2.7824 

-2.0381 

-1.2682 

-.4225 

.6857 

.8218 

.9632 

1.1098 

1.2615 

1.4168 

1.5762 

1,7392 

1.9057 

2.0753 

2.2475 

2.4221 

2.5988 

2.7769 

2.9564 

3.1372 

3.3192 

3.5024 

3.6868 

3.8720 

4.0580 

4.2446 

4.4318 

4.6195 

4.8076 

4.9962 

5.1851 

5.3742 

5.5636 

5.7532 

5.9431 

6,1332 

6.3236 

6,5142 

6.7049 

6.8958 

7.0869 

7.2781 


4.1983 
4.3691 
4.5403 
4.7118 | 
4.8836 ! 
5.0557 
5.2281 
5.4007 
5.5735 
5.7465 
5.9197 
6.0931 
6.2666 


2.1535 
1.3578 
-.5023 
.5439 
.6665 
.7930 
.9229 
1.0557 
1.1912 
1.3296 
1.4709 
1.6147 
1.7608 
1.9087 | 
2.0578 1 
2.2085 
2.3605 
2.5138 
2.6681 
2.8234 
2.9795 
3.13V* 
3.2938 
3.4519 
3.6105 
3.7696 
3.9292 
4.0891 
4.2493 
4.4099 
4.5707 
4,7318 
4.8931 
5.0547 
5.2164 
5.3784 
5.5405 
5.7027 
5.8651 
6.0276 
6.1903 


-3.7586 

-2.9816 

-2.1962 

-1.3910 

-.5312 

.5000 

.6190 

.7412 

.8664 

.9946 

1.1256 

1.2591 

1.3945 

1.5321 

1.6715 

1.8127 

1.9558 

2.1003 

2.2460 

2.3923 

2.53% 

2.6878 

2.8367 

2.9863 

3.1366 

3.2873 

3.4386 

3.5903 

3.7424 
3.8948 
4.0476 
4.2006 
4.3539 
4.5075 
4.6613 
4.8152 
4.%94 
5,1237 
5.2782 
5,4328 
5.5876 

5.7425 
5.8975 


-3.8146 

-3.0276 

-2.2327 

-1.4192 

-.5548 

.4657 

.5822 

.7014 

.8234 

.9481 

1.0750 

1.2043 

1.3358 

1.4693 

1.6044 

1.7412 

1.8792 

2.0184 

2.1589 

2.3003 

2.4426 

2.5857 

2.72% 

2.8740 

3.0189 

3.1643 

3.310i 

3.4563 

3.6029 

3,7499 

3,8971 

4.0446 

4.1924 

4.3404 

4.4886 

4.6870 

4,7855 

4.9342 

5.0831 

5,2321 

5.3812 

5.5305 

5.6798 


-2.2640 
-1.4435 
-.5752 
.4373 
.5518 
.6689 
.7885 
.9105 
1.0347 
1.1610 
1.2894 
1.41% 
1.5512 
1.6843 1 
1.8186 
1.9542 
2.0908 
2.2283 
2.3666 
2.5057 
2.6454 
2.7857 
2.9265 
3.0678 
3.2095 
3.3515 
3.4940 
3.6367 
3.7797 
3.9230 
4.0665 
4.2102 
4.3541 
4.4982 
4.6425 
4.7869 
4.9314 
5.0761 
5.2209 


-3.9045 
-3.1016 
-2.2915 1 
-1.4647 
-.5927 
.4137 
.5267 
.6420 
.7598 
.8797 
1.0016 , 
1.1256 
1.2514 
1.3789 
1.5G78 
1.6381 
1.7695 
1.9021 
2.0357 
2.1701 
2.3052 
2.4411 
2.5776 
2.7147 
2.8522 
2.9902 
3.1285 
3.2672 
3.4063 
3.5456 
3.6852 
3.8251 
3.9652 
4.1054 
4.2459 
4.3865 
4.5273 
4.6683 
4.8093 
4.9505 
5.0918 
5.2332 
5.3747 





















































Sample size, N 

13 

14 

15 

16 

17 

18 

19 

5.3059 

5.2241 

5.1534 

5.0916 

5.0369 

4.9879 

4.9438 

5.4423 

5.3585 

5.2860 

5.2226 

5.1666 

5.1164 

5.0712 

5.5788 

5.4929 

5.4187 

5.3538 

5.2964 

5.2449 

5.1987 

5.7153 

5.6274 

5.5514 

5.4850 

5.4263 

5.3736 

5.3263 

5.8519 

5.7620 

5.6842 

5.6162 

5.5562 

5.5022 

5.4539 

5.9886 

5.8966 

5.8171 

5.7475 

5.6861 

5.6310 

5.5815 

6.1253 

6.0313 

5.9500 

5.8789 

5.8161 

5.7598 

5.7092 

6.2621 

6.1660 

6.0829 

6.0103 

5.9461 

5.8886 

5.8369 

6.3989 

mil! 1 

6.2159 

6.1418 

6.0762 

6.0174 

5.9647 

6.5358 

6.4356 

6.3490 

6.2733 

6.2064 

6.1463 

6.0925 

6.6727 

6.5704 

6.48?* 

6.4048 

6.3365 

6.2743 

6.2203 

6.8096 

6.7053 

6.6152 

6.5364 

6.4667 

6.4043 

6.3482 

6.9466 

6.8402 

6.7483 

6.6680 

6.5970 

6.5333 

6.4761 

7.0836 

6.9752 

6.8815 

6.7996 

6.7272 

6.6623 

6.6040 

7.2207 

7.1102 

7.0147 

6.9313 

6.8575 

6.7914 

6.7320 

7.3578 

7.2452 

7.1480 

7.0630 

6.9878 

6.9205 

6.8600 

7.4949 

7.3803 

7.2813 

7.1947 

7.1182 

7.0496 

6.9880 

7.6321 

7.5154 

7.4146 

7.3264 

7.2486 

7.1787 

7.1160 

7.7693 

7.6505 

7.5479 

7.4582 

7.3790 

7.3079 

7.2441 

7.9065 

7.7856 

7.6813 

7.5900 

7.5094 

7.4371 

7.3722 

8.0437 

7.9208 

7.8146 

7.7218 

7.63 8 

7.5663 

7.5003 

8.1809 

8.0560 

7.9480 

7.8537 

7.7703 

7.6955 

7.6284 

8.3182 

8.1912 

8.0815 

7.9855 


7.8248 

7.7566 

8.4555 

8.3264 

8.2149 

8.1174 

8.0313 

7.9540 

7.8847 

8.5928 

8.4617 

8.3484 

8.2493 

8.1618 


8.0129 

8.7302 

8.5969 

8.4818 

8.3812 

8.2924 

8.2126 

8.1411 

8.8675 

8.7322 

8.6153 

8.5131 

8.4229 

8.3420 

8.2693 

9.0049 

8.8675 

8.7488 

8.6451 

8.5535 

8.4713 

8.3975 

9.1423 

9.0028 

8.8824 

8.7771 

8.6841 

8.6006 

8.5258 

9,2797 

9.1382 

9.0159 

8.9090 

8.8147 

8.7300 

8.6540 

9.4171 

9.2735 

9.1495 

9.0410 

8.9453 

8.8594 

8.7823 

9.5546 

9.4089 

9.2830 

9.1730 

9.0759 

8.9888 

8.9106 

9.6920 

9.5443 

9.4166 

9.3051 

9.2065 

9.1182 

9.0389 

9.8295 

9.6797 

9.5502 

9.4371 

9.3372 

9.2476 

9.1672 

9,9670 

9.8151 

9.6838 

9,5691 

9.4679 

9.3770 

9.2955 

10.1045 

9.9505 

9.8175 

9.7012 

9.5985 

9.5064 

9.4238 

10.2420 

10.0859 

9.9511 

9.8333 

9.7292 

9,6359 

9.5521 

10.3795 

10.2213 

10.0847 

9.9653 

9.8599 

9.7653 

9.6805 

10.5170 



10.0974 

9.9906 

9.8948 

9.8088 

10.6546 

■359 

Uiikll 

10.2295 

10.1213 

10.0243 

8.8372 

10.7921 

10.6277 

10.4857 

10.3616 

10.2521 

10.1538 

10.0656 

10.9297 

10.7632 

10.6194 

10.4938 

10.3828 

10.2833 

10.1940 

11.0672 

10.8987 

10.7531 

10.6259 

10.5135 

10.4128 

10.3223 


20 


4.9040 

5.0305 

5.1570 

5.2835 

5.4101 

5.5368 

5.6635 

5.7902 

5.9170 

6.0438 

6.1707 

6.2975 

6.4245 

6.5514 

6.6784 

6.8054 

6.9324 

7,0594 

7.1865 

7.3136 

7.4407 

7.5678 

7.6960 

7.8221 

7.9493 

8.0765 

8.2037 

8.3309 

8.4582 

8.5854 

8.7127 

8.8400 

8.9672 

9.0945 

9.2219 

9.3492 

9.4765 

9.6038 

9.7312 

9.8585 

9.9859 

10.1133 

10.2407 
























Sample size, N 


Safety Probability, 

margin P x 


Sm 

- 5.0 

- 4.0 

- 3.0 

- 2.0 

- 1.0 

0 

.1 

.2 

.3 

.4 

.5 

.6 

.7 

.8 

.9 

1.0 

1.1 

1.2 

1.3 

1.4 
1.3 
1.6 
1.7 


0 

0 

.0013 

.0227 

.1586 

.5000 

.5398 

.5792 

.6179 

.6554 

.6914 

.7257 

.7580 

.7881 

.8159 

.8413 

.8643 

.8849 

.9031 

.9192 

.9331 

.9452 

.9554 


1.8 

.9640 

1.9 

.9712 

2.0 

.9772 

2.1 

.9821 

2.2 

.9860 

2.3 

.9892 

2.4 

.9918 

2.5 

.9937 

2.6 

.9953 

2.7 

.9965 

2.8 

.9974 

2.9 

.9981 

3.0 

.9986 

3.1 

.9990 

3.2 

.9993 

3.3 

.9995 

3.4 

.9996 

3.5 

.9997 

3.6 

.9998 

3.7 

.9998 


21 


- 4.1579 

- 3.3100 

- 2.4568 

- 1.5917 

-.6954 

.2893 

.3956 

.5035 

.6130 

.7239 

.8362 

.9497 

1.0644 

1.1802 

1.2969 

1.4146 

1.5331 

1.6524 

1.7723 

1.8927 

2.0137 

2.1352 

2.2571 

2.3794 

2.5021 

2.6250 

2.7482 

2.8716 

2.9953 

3.1192 

3.2433 

3.3676 

3.4920 

3.6165 

3.7412 

3.8660 

3.9909 

4.1160 

4.2411 

4.3663 

4.4916 

4.6169 

4.7423 


22 


- 4.1740 

- 3.3233 

- 2.4673 

- 1.5998 

-.7017 

.2821 

.3882 

.4958 

.6049 

.7153 

.8271 

.9402 

1.0543 

1.1695 

1.2857 

1.4028 

1.5207 

1.6392 

1.7585 

1.8783 

1.9985 

2.1193 

2.2405 

2.3621 

2.4839 

2.6061 

2.7286 

2.8513 

2.9742 

3.0973 

3.2206 

3.3441 

3.4677 

3.5915 

3.7154 

3.8394 

3.9635 

4.0877 

4.2120 

4.3364 

4.4609 

4.5855 

4.7101 


23 


- 4.1893 

- 3.3358 

- 2.4772 

- 1.6073 

-.7077 

.2755 

.3813 

.4886 

.5973 

.7074 

.8188 

.9314 

1.0450 

1.1597 

1.2754 

1.3919 

1.5092 

1.6271 

1.7457 

1.8649 

1.9846 

2.1047 

2.2252 

2.3461 

2.4673 

2.5888 

2.7105 

2.8325 

2.9547 

3.0772 

3.1998 

3,3225 

3.4454 

3.5685 

3.6916 

3.8149 

3.9383 

4.0618 

4.1854 

4.3090 

4.4328 

4.5566 

4.6805 


24 


- 4.2036 
- 3 . 34^6 
- 2.4865 
- 1.6145 
-.7133 
.2694 
.3749 
.4819 
.5903 
.7000 
.8110 
.9232 
1.0364 
1.1506 
1.2658 
1.3818 
1.4985 
1.6159 
1.7340 
1.8526 
1.9716 
2.0911 
2.2111 
2.3313 
2.4519 
2,5728 
2.6939 
2.8152 
2.9368 
3.0586 
3.1805 
3.3026 
3.4249 
3.5472 
3.6697 
3.7924 
3.9151 
4.0379 
4.1608 
4.2838 
4.4068 
4.5299 
i 4.6531 


25 


- 4.2172 

- 3.3587 

- 2.4954 

- 1.6212 

-.7186 

.2636 

.3689 

.4757 

.5837 

.6931 

.8037 

.9155 

1.0283 

1.1421 

1.2568 

1.3724 

1.4886 

1.6055 

1.7231 

1.8411 

1.9596 

2.0786 

2.1979 

2.3176 

2.4376 

2.5579 

2.6784 

2.7992 

2.9202 

3.0413 

3.1627 

3.2842 

3.4058 

3.5276 

3.6495 

3.7714 

3.8935 

4,0157 

4.1380 

4.2603 

4.3828 

4.5053 

4.6278 


26 


- 4.2300 
- 3.3693 
- 2.5037 
- 1.6276 
-.7235 
.2582 
.3633 
.4698 
.5776 
.6867 
.7970 
.9084 
1.0208 
1.1342 : 
1.2485 
1.3636 
1.4794 
1.5959 
1.7129 
1.8305 
1.9485 
2.0669 
2.1857 
2.3049 
2.4244 
2.5441 
2.6641 
2.7843 
2.9047 
3.0253 
3.1461 
3.2670 
3.3881 
3.5093 
3.6306 
3.7520 
3.8735 
3.9951 
4.1168 
4.2386 
4.3604 
4.4823 
5.6043 


27 


- 4.2422 
- 3.3793 
- 2.5116 
- 1.6336 
-.7283 
.2531 
.3580 
.4643 
.5719 
.6807 
.7906 
.9017 
1.0138 
1.1269 
1.2407 
1.3554 | 
1.4708 
1.5868 
1.7034 
1.8205 
1.9380 
2.0560 
2,1743 
2.2930 
2.4120 
2.5312 
2.6507 
2.7704 
2.8903 
3.0104 
3.1306 
3.2510 
3.3716 
3.4922 
3.6130 
3.7339 
3.8549 
3.9759 
4.0971 
4.2183 
4,3396 
4.4610 
4.5824 


28 


- 4.2538 
- 3.3888 
- 2.5192 
- 1.6393 
-.7327 
.2483 
.3530 
.4591 
.5664 
.6750 
.7847 
.8955 
1.0072 
1.1199 
1.2334 
1.3477 
1.4627 
1.5783 
1.6945 
1.8112 
1.9283 
2.0458 
2.1637 
2.2819 
2.4004 
2.5192 
2.6382 
2.7574 
2.8768 
2.9964 
3.1162 
3.2361 
3.3561 
3.4763 
3.5966 
3.7170 
3.8374 
3.9580 
4.0786 
4.1994 
4.3201 
4.4410 
4.5619 ] 
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TABLE A-5.— Continued. 


<c) Concluded. 


Sample size, N 


4.8678 
4.9934 
5.1190 
5.2447 
5.3704 
5.4961 
5.6219 
5.7478 
5.8737 
5.99% 
6. 1255 
6.2515 
6.3775 
6.5035 
6.6296 
6.7557 
6.8818 
7.0080 
7 1341 
7.2603 
7.3865 
7.5127 
7.6390 
7.7652 
7.8915 
8.0178 
8.1441 
8.2704 
8.3967 
8.5231 
8.6494 
8.7758 
8.9022 
9.0285 
9.1549 
9.2814 
9.4078 
9.5342 
9.6606 
9.7871 
9.9135 
10.0400 
10.1665 I 


4.8348 

4.9595 

5.0843 
5.2091 
5.3340 
5.4590 
5.5840 
5.7090 
5.8340 
5.9591 

6.0843 
6.2094 
6.3346 
6.4598 
6.5851 
6.7103 
6.8356 
6.9609 
7.0863 
7.2116 
7.3370 
7.4624 
7.5878 
7.7132 
7.8387 
7.9641 
8.08% 
8.2151 
8.3406 
8.4661 
8.5916 
8.7172 
8.8427 
8.9683 
9.0938 
9.2194 
9.3450 
9.4706 
9.5962 
9.7218 
9.8474 
°.9/30 
10.0987 


4.8044 

4.9284 

5.0524 

5.1765 

5.3007 

5.4259 

5.5491 

5.6734 

5.7977 

5.9220 

6.0464 

6.1708 

6.2952 

6.4197 

6.5442 

6.6687 

6.7932 

6.9178 

7.0423 

7.1669 

7.2916 

7.4162 

7.5408 

7.6655 

7.7902 

7.9149 

8.03% 

8.1643 

8.2891 

8.4138 

8.5386 

8.6633 

8.7881 

8.9129 

9.0377 

8.1625 

9.2873 

9.4122 

9.5370 

9.661° 

9.7867 

9.9116 

10.0365 


4.7764 

4.8997 

5.0231 

5.1465 

5.2699 

5.3934 

5.5170 

5.6405 

5.7641 

5.8878 

6.01 15 

6.1352 

6.2589 

6.3827 

6.5065 

6.6303 

6.7541 

6.8780 

7.0019 

7.1257 

7.2497 

7.3736 

7.4975 

7.6215 

7.7455 

7.8695 

7.9935 

8.1175 

8.2415 

8.3656 

8.4897 

8.6137 

8.7378 

8.8619 

8.9860 

8.1101 

9.2342 

9.3583 

9.4823 

9.6066 

9.7308 

9.8549 

9.9791 


4.7504 

4.8731 

4.9958 

5.1186 

5.2414 

5.3643 

5.4872 

5.6101 

5.7331 

5.8561 

5.9791 

6.1022 

6.2253 

6.3484 

6.4716 

6.5947 

6.7179 

6.8411 

6.9644 

7.0876 

7.2109 

7.3342 

7.4575 

7.5808 

7.7041 

7.8275 

7.9508 

8.0742 

8.1976 

8.3210 

8.4444 

8.5678 

8.6912 

8.1847 

8.938! 

9.0616 

9.1850 

°.30«5 

9.4320 

9.5555 

9.6790 

9.8025 

9.9260 


4.7263 

4.8484 

4.9706 

5.0927 

5.2150 

5.3372 

5.4595 

5.5819 

5.7043 

5.8267 

5.9491 

6.0716 

6.1941 

6.3166 

6.4391 

6.5617 

6.6843 

6.8069 

6.9295 

7.0522 

7.1749 

7.2975 

7.4202 

7.5430 

7.6657 

7.7884 

7.9112 

8.0340 

8.1567 

8.2795 

8.4023 

8.5252 

8.6480 

8.7708 

8.8937 

9.0165 

9.1394 

8.2622 

9.3851 

9.5080 

9.6309 

9.7538 

9.8767 


4.7039 

4.8254 

4.9470 

5.0686 

5.1903 

5.3120 

5.4338 

5.5556 

5.6774 

5.7992 

5.9211 

6.0430 

6.1650 

6.2869 

6.4089 

6.5309 

6.6530 

6.7750 

6.8971 

7.0192 

7.1413 

7.2634 

7.3856 

7.5077 

7.6299 

7.7521 

7.8743 

7.9%5 

8.1187 

8.2409 

8.3632 

8.4854 

8.6077 

8.7300 

8.8522 

8.9745 

9.0%8 

9.2191 

9.34 !4 

9.4638 

9.5861 

9.7084 

9.83(8 


4.6829 

4.8039 

4.9250 

5.0461 

5.1673 

5.2885 

5.4097 

5.5310 

5.6523 

5.7736 

5.8950 

6.0164 

6.1378 

6.2592 

6.3807 

6.5022 

6.6237 

6.7452 

6.8668 

6.9883 

7.1099 

7.2315 

7.3531 

7.4748 

7.5964 

7.7181 

7.8398 

7.9614 

8.0831 

8.2048 

8.3266 

8.4483 

8.5700 

8.6918 

8.8135 

8.9353 

9.0571 

9.1788 

9.3006 

9.4224 

9.5442 

9.6660 








Probability, 

P* 


Sample size, N 



30 

40 

50 

60 

70 

80 

90 

100 

0 

- 4.2753 

- 4.3596 

- 4.4191 

- 4.4640 

- 4.4996 

- 4.5286 

BH 


0 

- 3.4065 

- 3.4757 

- 3.5245 

- 3.5613 

- 3.5905 

- 3.6142 


IBRgjQ 

.0013 

- 2.5332 

- 2.5879 

- 2.6264 

- 2.6555 

- 2.6785 

- 2.6973 


IBS?™ 

.0227 

- 1.6500 

- 1.6916 

- 1.7208 

- 1.7427 

- 1.7601 

- 1.7742 

- 1.7861 

- 1.7962 

.1386 

-.7411 

-.7732 

-.7954 

-.8121 

-.8251 

-.8358 

-.8446 

-.8522 

.5000 

,2394 

.2061 

.1837 

.1673 

.1547 

.1445 

.1361 

.1290 

,5398 

.3439 

.3094 

.2864 

.2696 

.2566 

.2462 

.2376 

.2304 

.5792 

.4496 

.4138 

.3901 

.3727 

.3594 

.3487 

.3399 

.3325 

.6179 

.5565 

.5193 

.4946 

.4767 

.4629 

.4519 

.4428 

.4352 

.6554 

.6646 

.6257 

.6000 

.5814 

.5671 

.5557 

.5464 

.5385 

.6914 

.7737 

.7331 

.7063 

.6869 

.6721 


.6505 

.6424 

.7257 

.8840 

.8414 

.8134 

.7931 

.7777 

.7654 

.7553 

.7468 

.7580 

.9951 

.9505 

.9211 

.9000 

.8839 

.8710 

.8605 

.8517 

.7881 

1.1072 

1.0603 

1.0296 

1.0075 

.9906 

.9773 

.9663 

.9571 

.8159 

1.2201 

1.1708 

1.1386 

1.1155 

1.0979 

1.0839 

1.0725 

1.0630 

.8413 

1.3337 

1.2820 

1.2482 

1.2240 

1.2056 

1.1911 

1.1791 

1.1692 

.8643 

1.4480 

1.3937 

1.3583 

1.3330 

1.3138 

1.2986 

1.2861 

1.2757 

.8849 

1.5628 

1.5059 

1.4689 

1.4424 

1.4223 

1.4064 

1.3935 

1.3826 

.9031 

1.6782 

1.6186 

1.5799 

1.5522 

1.5312 

1.5146 

1.5011 

1.4898 

.9192 

1.7941 

1.7317 

1.6912 

1.6623 

1.6404 

1.6231 

1.6090 

1.5972 

.9331 

1.9105 

1.8452 

1.8029 

1.7727 

1.7499 

1.7319 

1.7171 

1.7049 

.9452 

2.0272 

1.9590 

1.9149 

1.8834 

1.8596 

1.8408 

1.8255 

1.8127 

.9554 

2.1442 

2.0731 

2.0271 

1.9944 

1.9696 

1.9500 

1.9341 

1.9208 

.9640 

2.2616 

2.1875 

2.1396 

2.1055 

2.0797 

2.0594 

2.0428 

2.0290 

.9712 

2.3793 

2.3021 

2.2523 

2.2169 

2.1901 

2.1689 

2.1517 

2.1374 

.9772 

2.4972 

2.4170 

2.3652 

2.3284 

2.3006 

2.2786 

2.2608 

2.2459 

.9821 

2.6154 

2.5320 

2.4782 

2.4401 

2.4112 

2.3885 

2.3700 

2.3545 

.9860 

2.7337 

2,6472 

2.5915 

2.5519 

2.5220 

2.4984 

2.4792 

2.4633 

.9892 

2.8523 

2.7626 

2.7049 

2.6039 

2.6329 

2.6085 

2.5887 

2.5721 

.9918 

2.9710 

2.8782 

2.8184 

2.7759 

2.7439 

2.7187 

2.6982 

2.6810 

.9937 

3.0899 

2.9938 

2.9320 

2.8881 

2.8550 

2.8290 

2.8078 

2.7901 

.9953 

3.2089 

3.1096 

3.0457 

3.0004 

2.9663 

2.9393 

2.9174 

2.8992 

.9965 

3.3280 

3.2255 

3.1596 

3 . M 28 

3.0776 

3.0498 

3.0272 

3.0084 

.9974 

3.4473 

3.3416 

3.2735 

3.2253 

3.1889 

3.1603 

3.1370 

3.1176 

.9981 

3.5667 

3.4577 

3.3875 

3.3378 

3.3004 

3.2709 

3.2469 

3.2269 

.9986 

3.6861 

3.5738 

3.5016 

3.4505 

3.4119 

3.3815 

3.3568 

3.3362 

.9990 

3.8057 

3.6901 

3.6158 

3.5631 

3.5234 

3.4922 

3.4668 

3.4456 

.9993 

3.9253 

3.8064 

3.7300 

3.6759 

3.6351 

3.6030 

3.5768 

3 . 555 ! 

.9995 

4.0451 

3.9228 

3.8443 

3.7887 

3.7467 

3.7137 

3.6869 

3.6646 

OOCWC 

.5WO 

4.1649 

4.0393 

3,9586 

3.9015 

3.8585 

3.8246 

3.7970 

* 3.7741 

.9997 

4.2847 

4.1558 

4.0730 

4.0144 

3.9702 

3.9355 

3.9072 

3.8837 

.9998 

4.4047 

4.2724 

4.1874 

4.1273 

4.0820 

4.0464 

4.0174 

3.9933 

.9998 

4.5247 

4.3891 

4.3019 

4.2403 

4.1939 

4.1573 

4.1276 

4.1029 






































Appendix B 

Project Managers Guide on Product Assurance 


This concise, practical appendix on product assurance 
management aims to convince you that reliability and quality 
I assurance are major components of project success. It is 
{ especially useful to newly appointed project managers and 
1 others concerned with specifying product assurance provisions. 

I jt begins with a general discussion of the product assurance 
[ manage r and his or her roles, duties, and functions and 
1 then provides condensed descriptions, with illustrations, of 
i frequently applied reliability and quality assurance re- 
1 quirements. NASA NHB 5300.4 and Department of Defense 
j Mil -STD-785 series documents (refs. B-l to B-ll) cover 
the same subjects. 


' Product Assurance Manager 

! Hob.— Product assurance managers in NASA Lewis’ Office 

j of Reliability and Quality Assurance advise the various project 
offices on R&QA matters. Their leadership is extremely 
; important during the preparation of a project plan, the 
i generation of a statement of work, the review of a bidder’s 
| proposals, and the final contract negotiations. The assigned 
{ product assurance manager is normally included on the project 
organization chart in a staff reporting position. A product 
■ assurance manager works closely with the project office that 
j he or she is supporting to develop R&QA requirements that 
t are j„ consonance with the uniqueness of the project and that 
. are significantly cost effective. 

* Responsibilities. -The product assurance manager supports 
| projects by providing technical management leadership in 
\ applying R&QA principles to the design, manufacture, test, 

' handling, installation, and operation of aeronautics, space, and 
I energy projects. To accomplish this duty , he or she performs 
[ the following functions: 

1 (1) Formulates R&QA requirements for assigned projects, 

i (2) Incorporates appropriate N ASA NHB 5300 A or M1L- 
j STD-785 series (refs. B-l to B-ll) requirements into 
A statements of work. 

(3) Evaluates proposals and then participates in contract 
I negotiations. 

I (4) Serves on source evaluation boards when assigned. 


(5) Prepares letters of delegation for R&QA functions and 

mandatory inspection points to cognizant Government inspec- 
tion agencies. . 

(6) Review: '•d evaluates R&QA plans, fabrication and test 

inspection procedures, process specifications, failure reports, 
corrective actions, equipment history records, and odier docu- 
ments relating to R&QA. 

(7) Monitors activities of contractor and Government 
inspection agencies to assure compliance with R&QA 
requirements. 

(8) Arranges and coordinates problem investigations and 
analyses with interdirectorate reliability and quality engi- 
neering support groups. 

(9) Attends directorate and project management meetings. 
10) Supports project design reviews and program status 

meetings. 

(11) Supports the project in the final acceptance of equip- 
ment, when planned. 


Economics of R&QA 

Classical curves (fig. B-l) show the relationship of product 
quality cost to operational cost. When the percentage of defects 
is low, the product quality cost is extremely high. Conversely, 
when the percentage of defects is high, the operational cost 
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Figure B-I.-Relatlomhip of product quality cost to operational cos.. 
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TABLE B- 1. -RELIABILITY AND QUALITY ASSURANCE REQUIREMENTS IMPOSED 
ON VARIOUS PROGRAM TYPES 


[Composite wind itirbine blades, C; global air sampling program, G; lift/cruise fan, L; materials for advanced turbi.ie engines, 
M; electrical powe< processor, P; quiet, clean, short-haul experimental engine, Q; JT8D refan engines, R; space experiments, 
S; variable-cycle engine, V; 200-kW wind turbine generators, W.] 
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is extremely high. The intersection of these curves gives the 
optimui.i goal from a cost viewpoint. 

The product assurance manager has the optimum cost goal 
in mind when selecting the R&QA program requirements. 
However, there may be some critical items, from an engineer- 
ing viewpoint, where additional safeguards m»st be established, 
and the need for close R&QA control is mandatory. Under 
such a condition economics is still a major consideration. 

Development of R&QA Requirements 

Reliability and quality assurance are broad and diverse 
disciplines that have some overlapping authority with pro- 
curement, engineering, manufacturing, and testing. This 
ovsrlap problem is lessened at Lewis by assigning a product 


assurance manager at the beginning of each project phase when 
R&QA requirements must be formulated. 

The product assurance manager is qualified to sell the need 
for R&QA controls. He or she has the proper skills, training, 
and projec ‘ experience to work out the various organizational 
relationships and can tailor the many R&QA tasks in the NHB 
5300.4 or MIL-STD-785 series documents (refs. B-l to 
B-l 1) into something realistic, reasonable in scope, and easily 
understood. In addition, the product assurance manager 
ultimately has the responsibility for assuring that the R&QA 
program is consistent with the project objectives and that the 
program will satisfy the overall mission requirements. As an 
example, table B-l lists the actual requirements imposed on 
10 Lewis contracts. The particular project phase associated 
with each contract is also identified. 
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Parts Selection and Screening 

The costs incurred during subsystem and system testing are 
inversely proportional to the money that is spent for examining 
and testing the parts. Success is directly related to the part 
screening costs. For example, the exceptional operational life 
of the Space Electric Rocket Test (SERT) n satellite is no doubt 
attributable to the extensive parts selection and screening 
program. 


Other factors influence parts selection and screening, such 
as the criticality of the hardware application, unusual envi- 
ronments, contractor experience, and in-house resources (R&QA 
parts screening laboratory, etc.). The selection can range from 
a high-reliability part (identified in a Government- or industry- 
preferred parts handbook) to an off-the-shelf commercial part 
(fig. B-2). Likewise, screening is a selective process as called 
out in the source control document. Reference B-6, paragraph 
IF302, explains in detail how screening can be done. 


1. PART USED IN (ASSEMBLY, COMPONENT, AND SYSTEM) j 

TRANSMITTER EQUIPMENT PACKAGE (TEP) 


2. LeRC REQUESTER, CONTRACTOR, AND PROJECT 

3. CONTRACT NO. (IF APPLICABLE) 

TRW Systems 

NAS3-I5839 


k. DESCRIPTION OF PART 

Hybrid Driver, High Voltage, High Current 


5. DRAWING SPEC NO. 
PTA-AHS 


6. PART NO. 

PTA-4IA5-0II 


7. NFR. AND MFR'S EQUIVALENT 
COMMERCIAL PART STYLE 
DESIGNATION 
National 
ohoooSh 


8. 

1 

PREVIOUS APPROVAL (AGENCY) 

NASA 

SAMSO/USAF 

1 

FOR USE IN 

Pioneer 

FLTSATCOM 

ON CONTRACT 

TRW 

TRW 

9. 

COMPARISON BETWEEN NON-STANDARD PART AND STANDARD PART: 

Standard part not available. This part selected based on previous successful 
use at TRW In similar applications. 

10. 

TEST DATA AND APPLICATION INFORMATION 




Part qualified by Group BSC testing on production lot. 


11. 

Tills PART SHOULD BE CONSIDERED FOR INCLUSION INTO THE NSPL(MIL-STD-975) 
STATE REASONS FOR RECOMMENDATION ON REVERSE SIDE 


12 . 


CONTRACTOR CERTIFICATION 

I CERTIFY THAT, TO THE BEST OF MY KNOWLEDGE, THE ABOVE INFORMATION AND DATA 
ARE CORRECT. 


PARTS OR RELIABILITY ENGINEER (SIGNATURE) 

PROJECT MANAGER OR DESIGNATED REPRESENTATIVE 


_DATE_ 

DATE 


13. FOR LeRC USE ONLY 


APPROVALS 


THE PARTS BRANCH (DOES,. DOES NOT) CONCUR 
WITH THE USE CF THIS PART 


SIGNATURE 


DATE 


THIS REQUEST (IS, IS NOT) APPROVED BY THE 
PROJECT MANAGER OR DESIGNATED REPRESENTATIVE 
PROJECT DIRECTOR (IF REQUIRED) 

DIRECTOR OF SYSTEMS RELIABILITY (IT REQUIRED) 


Figure B-2. —Typical nonstandard pels approval request 



Identification of Parts and Materials 


Material Certification 


It is good v.igineering practice to identify parts, components, 
and materials with a part number, a serial number, and a date 
code, as applicable. Furthermore, the marking on parts and 
components should be affixed in a location that is easily seen 
when the item is installed on an assembly. The identification 
method (paint, electrochemical, etc.) and location on the item 
are included on a drawing, a specification, or other associated 
engineering document (fig. B-3, note 6). During the period 
of fabrication, assembly, and testing the system of marking 
, and recordkeeping will provide a way of tracing backward 
' from an end item to the part or material level. 


There are applications in which the certification of metallic 
and nonmetallic materials is essential to assure that the 
chemical and physical properties of the materials are compat- 
ible with the design requirements. Once a material is selected 
by the engineer and precisely defined by a specification 
(Federal, Society of Automotive Engineers, American Society 
for Testing and Materials, etc.), the purchase order for 
materials such as steels, aluminum alloys, brass, welding rods, 
solder, metal coatings, gases, and potting compounds should 
require that a test report, a certificate of conformance (fig. 
B-4), or both accompany the vendor s shipment. In addition 
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1. MATERIAL SPECIFICATION: ALUMINUM ALLOY PLATE AND ERECT 
PER FEDERAL SPECIFICATION QQ-A-250/BF (5003*0). 

2. ULTRASONIC INSPECT PER MIL-U-01065 QUALITY LEVEL II PRIOR 
TO ANY FABRICATION PROCESSES. 

3. ^JKaLL OVER PER ANSI »A0. 1-1962 UNLESS OTHERWISE N01»* 

0. THREADS SNALL BE PER MIL-S-77N2B. 

5. FLUORESCENT PENETRANT INSPECT PER X1L-T-60660 TYPE I. 
METHOD A. WATER WASHABLE. USE TRACER-TECH PENETRANT 
NO* P-130 Off APPROVED EQUIVALENT* NO CRACKS ALL0WE9. 

6. IDENTIFICATION MARKING SHALL BE DONE BY THE MONQDE ELECTRO- 
LYTE PROCESS. OR OTHER ACCEPTABLE ELECTRO-CHEW CAL- 
ETCMIHG PROCESS. NUMBERS AND LETTERS SHALL BE APPROXI- 
MATELY *1 HIGH. 


7. 


a. 


0. 


10 . 


II. 


AFTER THE FINAL ASSEMBLY FIT-UP HAS BEEN CCMPLETEQ 10 
ACCORDANCE WITH DWG. CO 634718 AHO PRIOR TO FINAL 
ASSEMBLY OF THIS PART. THE FOLLOWING PROCEDURE SHALL 5E 
'.DHERED TO: 

A. CLEAN ALL SURFACES WITH TRICHLOROETHYLENE . TYPE H 
PEN FEOERAL SPECIFICATION 0-1-5346 AMEND* AIR 
DRY* 

B. ULTRA CONICALLY CLEAN IH FRE-R PER NASA SPEC NO- 73?!. 

C. HANOLE WITH CLEAN LINT-FREE NYLON GLOVES* 

0. APPLY CONTAMINATION BARRIER PER PARAGRAPH 3.3.2 OF 
HIL-M-9950 (U.S.A.F*) CONTAMINATION BARRIER SMALL 
BE FREE OF OILS ANO FOREIGN MATERIAL* SEAL AND 
IDENTIFY BY PART NO- ANO SERIAL NO/ ON TAO ATTACHED 


TO BAG* 

WEUKH6 GAS SHtttOEO TuNGSTCVl Wfc*. 

MIL * W* A604 . CAUTtOM ** OO WOT INStOt 

OF ftOOY TO M.COMC CONTAMINATED* 


VEtDlNG HOD GENERAL. ftUAUYY CONTROL j 

•fturiCATlOH ONLT PER QCL* R- 5G6 K > CLASSvWOWtJ 


CC 634736 


RAOtOBHAFHtC IHSrtCTlOH ft*. MIL-&TO-4S3 

RADIOBHAFHIC »N4PtCT»M AVIO ACCtHTANCt SHALL 
BC IN ACCORDANCE WITH MIL-R- AS 774 


Figure B -3.— Typical drawing specifications. 


to the vendor’s certification it may be necessary to perform 
periodic in-house testing of metallic and nonmetallic materials 
to assure their continued validity. 


Review of Drawings 

Before releasing the engineering drawings to the manufac- 
turer, design engineers may avail themselves of the.technicai 
services provided by quality engineers when developing 


CAST TECHNOLOGY INCORPORATED 


Mil EMC iOUlEVAiD 
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M 81V *Iiwi» Research C antor 
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AND 

MECHANICAL TESTS 
(Job 1365) 

HA8A-Iieiris Research center 
21000 Brockpark Road 
Cleveland# Ohio 44135 
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CTX-22 (11-66) 


Figure B-4.— Typical material unification. 
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specification callouts in the note section of the drawings (fig. 
B-3). Precise information on materials, surface finish, 
processing, nondestructive testing, cleanliness, identification, 
packaging , special instructions, etc., is important in obtaining 
a quality product. 


Changes in Engineering Documents 

Starting early in the design phase a system is established 
to control changes (fig. B-5) in engineering documents. 
Changes in released drawings, specifications, test procedures. 



Figure B-J. -Typical engineering change ordrr. 
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Figure B-6.-Typical failure mode and effect analysis. 


and rented documents can be critical, particularly during the 
building and testing phases. For this reason the latest 

arC ■ r ° CeSSed Carly ' and their d^^ution 
s expedited to the participating line organizations. In addition 

the system must provide for removing obsolete documents. 

Failure Mode, Effects, and Criticality Analysis 

The fundamental objective of a failure mode, effects, and 
criticality analysis Is to identify the critical failure areas in a 
design. In order to accomplish this, each functional component 
or higher level, if adequate to attain the intended purpose) 
^sequentially assumed to fail, and the broad effects ofeach 
such failure on the operation of the system (fig. B-6) are 

,ubj “' are ,v, "' bie in mil ' 


'Use the latest document that has been issued. 


Use of a Process Plan 

!t is good quality assurance practice to identify in a plan 
(f'g. B-7) those manufacturing operations that must be 
performed in a particular sequence. The most commonly used 
processes are machining, mechanical fastening, grinding 
brazing, welding, soldering, polishing, coating, plating, 
ra lography, ultrasonics, fluorescent penetrant inspection 
magnetic part'de inspection, painting, bonding, heat treating,’ 
identification marking, and safety wiring. 

Calibration of Measuring Devices 

The calibration of instruments is necessary where physical 
quantit.es are to be measured with any degree of accuracy 
The instruments considered, which use standard units of 
measure, include test and measuring instruments, various 
accessories, and gages. As defined herein, calibration includes 
repairing, periodic (recall) maintenance, and determining the 
accuracy (adjustments made as required) of the mSing 
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devices when compared with known standards from the 
National Institute of Standards and Technology. Figure B-8 
shows a ypical certificate of calibration. 


Inspection of Hardware Items 


Quality control inspectors check in-process items against 
acceptable quality standards and engineering documents (fig. 
6 - 9 ). Minor deviations from good quality practices are 
normally resolved at the work site; otherwise they are brought 
to the attention of the inspection supervisor. If the quality 
standard being violated is not contained in an engineering 
document, the supervisor may review the inspector’s decision 
if the risks are involved. If the discrepancy is a characteristic 
defined by an engineering document, die final decision is made 
by material review engineering and product assurance repre- 
sentatives or the material review board (engineering, product 
assurance, and Government representatives). 


Nonconforming Hardware 


When hardware is to be built, some provision must be made 
for the orderly review and disposition of all items that are 
determined by inspection or test not to conform to the drawing, 
specification, or workmanship requirements. The system most 
frequently used comprises two basic methods: 


(1) It provides for review and disposition of hardware that 
can be reworked into a conforming condition without an 
engineering change, an instruction, or both. Traditionally, an 
engineer cr a product assurance manager is authorized to make 
this decision. 

(2) If the item cannot be reworked to meet the engineering 
specifications, the material review board reviews the problem. 
This board consists of engineering, product assurance, 
and, when required, Government representatives. In difficult 
situations the board members are not reluctant to consult 
with other organizations and persons to arrive at the best 
decision. 
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WESTERN AUTOMATIC TEST SERVICES 

W1 CoOHMMMStfMt 

Ptlo Alio. California. 94303 

Mis) sat-eoM 


10:Utton Industries 
960 Industrial Way 
San Carlos, CA 


DATE: 21 July 1986 


Reference: Your Order Ho. 49721 

WATS Order No. 8526 
TO WHOM IT MAY CONCERN: 

The equipment listed below has been duly calibrated by Wavecoia Indus- 
tries AlATS Group per your Instructions. 

to the National Bureau of St andards to the extent A r 

Calibration facilities. C /JX ''/ J /] /) 


HAVE COM INDUSTRIES/WATS Group 


0 — »»*** 


Description 


Serial No, 


NA8A INPUT SYSTEM 
NASA INPUT SYSTEM 
ym OUTPUT 8YSTBM 
NASA OUTPUT SYSTEM 




Figure B-8.— Typical certificate of calibration. 
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Figure B-9.— Typical mandatory quality control inspection points. 





































Documenting Equipment Discrepancies 

Certain characteristics in a design are distinct, describable, 
and measurable in engineering units. The critical character- 
istics are generally identified by engineering documents and 
are closely controlled by quality assurance personnel. 

Whenever any characteristic is determined not to conform 
to released engineering requirements, one of the following two 
reporting methods applies: 

(1) A minor discrepancy is recorded in a discrepancy log 
(fig. B-10). A disposition must be made by an engineer, an 
inspector, or both if the condition is a minor discrepancy (e.g., 
a scratch on a metal surface or excess material) that does not 
adversely affect form, fit, or function and can be used “as 
is” or reworked to engineering requirements. 

(2) A failure discrepancy report is written. A disposition 
is obtained through the engineering review board (ERB) if a 
mechanical, electrical, or electronic system or subsystem has 
failed to perform within the limits of a critical characteristic 
identified by an engineering drawing, specification, i’st 
procedure, or related engineering document. 

Failure Analysis of Farts 

Some failed parts are analyzed and investigated to determine 
the cause of the failure (fig. B-l 1). Corrective action is worked 


out to assure that the problem does not recur. The corrective 
action is verified by testing. The problem is closed by ERB 
review. Sometimes corrective action may change a component 
application criterion, improve a packaging technique, or revise 
a test procedure. Often the detailed physical and chemical 
examination reveals that a refinement is needed in the materials 
used during the manufacturing of a part or that an improvement 
in the parts screening process is necessary. 

Quality Assurance Recording of Production, Inspection, 
and Test Operations 

Manufacturing, inspecting, testing, and related operations 
on major assemblies and subassemblies should be documented 
for several reasons. Such documentation can provide a status 
record of the work in progress a? well as the work completed. 
Also, it can become a part of the permanent record of 
production, inspection, and test operations. The sophistication 
of the format and the entries in the log can be adjusted to suit 
the type of contract— research, development, or production. 
These chronological entries in the log can be summarized and 
included in an acceptance data package, which contains 
information that is helpful to review during a contractor’s 
acceptance of a supplier’s equipment or during final Govern- 
ment acceptance of a contract end item. Figure B-l 2 shows 
a list used to check an item’s conformance to specifications. 
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Figure B-10. — Typical discrepancy log. 
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Figure B- 1 1.— Failure report, analysis, and corrective action flowchart. 











2.0 Quality assurance checklist for conformance to specifications of 
Communications Technology Satellite (CTS) output stage tube (QST) 

OSTS/N : 2021 Classification:, QTM-2(QF-2) 

2.1 Overall efficiency 


Specification: 50 percent 
minimum over CTS band of 
12.038 GHz to 12.123 GHz, 
at saturation 


Actual: 40.7 percent 
minimum at 12.040 GHz. 
Out of specification. 
(Waiver required.) 


2.2 Center frequency 


Specification: 12.0805 GHz 


Actual: 12.0805 GHz 




2.3 RF power output 


Specification: 200 W 

Actual: 170 W minimum 

minimum at saturation 

at 12.040 GHz. 

over CTS band of 12.038 to 

Out of specification. 

12.123 GHz 

(Waiver required) 


2.4 Small signal bandwidth 


Specification: 3dB 

Actual: 2.4 dB maximum 

maximum peak to peak 
measured at 10 dB below 
peak saturation over the 
CTS band, 12.038 to 12.123 GHz 

peak to peak 

& 


Figure B- 12. — Checklist for item conformance to specifications. 


Quality Assurance for Suppliers of Materials and Services 

Materials and services acquired by the user from outside 
sources must satisfy, as applicable, either contract, Govern- 
ment, or company reliability and quality assurance require- 
ments. The user needs a system of control that involves 

(1) Selecting acceptable or qualified sources 

(2) Performing surveys and audits of the supplier’s facilities 

(3) Inspecting supplier's products received 

(4) Taking corrective action on problems that occur 
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Appendix C 

Reliability Testing Examples 


A great deal of work has been done by various researchers 
to develop probabilistic methods suitable for reliability prob- 
lems (ref. C-l). Probabilistic methods that apply discrete and 
continuous random variables to user problems are not as well 
covered in the literature. 

This appendix concentrates on four useful functions: 
(1) failure /(f), (2) reliability /?(f), (3) failure rate X, and 
(4) hazard rate X'. Because we usually need to know how 
well a point estimate has been defined , some consideration 
is given to confidence intervals for these functions. The 
appendix also explains methods for planning events at the 
critical delivery milestone and closes with a brief explanation 
of two reliability case histories. 


The reliability function /?(/) is given by 
*(0 = 1 -£?(0 
In integral form £(0 is given by 


RU) = dt 
Differentiation y\ ids 


dR(t) dQ(t) _ 

dt dt 


Useful Distribution Functions 

The failure function /(/), which defines failures as a function 
of time or number uf cycles, is important knowledge obtained 
from reliability testing. Failure records are kept on a particular 
piece of hardware to obtain a histogram of failures against 
time. This histogram is studied to determine which failure 
distribution fits the existing data best. Once a function /(r) 
is obtained, reliability analysis can proceed. In many cases 
sufficient time is not available to obtain large quantities of 
failure density function data. In these cases experience can 
be used to determine which failure frequency function best 
fits a given set of data. Table C-l lists seven distributions— 
five continuous and two discrete. These distributions can be 
used to describe the time-to-failure functions for various 
components. The derivation of the four reliability functions 
tor the seven listed distributions is explained in the next section 
(ref. C-2). 

Derivation of Qft), R(t), X, and X' functions.— The 
unreliability function Q(t) is the probability that in a random 
trial the random variable is not greater than n hence, 

Q(t) = j 0 p(f) dt 

When time is the variable, the usual range is 0 to f, implying 
that the process operates for some finite time interval. This 
integral is used to define the unreliability function when failures 
are being considered. 


The a posteriori probability of failure pf in a given time 
interval, t\ to t 2 , can be calculated by using these equations 
and is given by 

= *<h J** <,) *] 

Substituting and simplifying gives 


Pf= 1 - 


R(h) 

*(/,) 


The rate at which failures occur in a time interval is defined 
as the ratio of the probability of failure in the interval to the 
interval length. Thus, the equation for failure rate X is given by 


} _ R(U)-RQ2) 

(r 2 - /,)!?(*,) 


1 

h -»i 



mi 


Substituting t { = t and t 2 = t + h into this equation gives 


R(t) -JW + h) m -fi(r + h) 
(f + ft - t)R(t) ~ hR(t) 


TABLE C-l.— FIT DATA FOR FAILURE FUNCTIONS 


Distribution 

Failure fit 

Continuous distribution 

Exponential 

Normal 

Weibull 

Gamma 
Log normal 

Complex electrical systems 
Mechanical systems subject to wear 
Mechanical, electromechanical, or electrical 
parts: bearings, linkages with fatigue loads, 
relays, capacitors, and semiconductors. 
Reduces to exponential distribution if a * /, 
0 * 1 , and y = 0 

Combined mechanical and electrical systems 
Mechanical parts under stress rupture loading 

Discrete distribution 

Poisson 

Binomial 

One-shot parts 

Complex electrical systems for probability 
of N f defects 


The instanleous failure rate in reliability literature is ofte n 
called the hazard rate. The hazard rate X ' is by definition the 
limit of the failure rate as h - 0. Using a previous equation 
and taking the limit of the failure rate as h - 0 gives 

_ lim R{t) - R(f + jj) 

X hR(t) 


Figure C-l (pp. 192 and 193) shows a summaiy cf the useful 
frequency functions for the failure distributions given in 
table C- 1 . These functions were derived by using the defining 
equations given previously. Choose any failure fiinction and 
verify that R(t), X, and X' are properly defined by going 
through the derivation yourself. Five reliability problems using 
the continuous distributions given in figure C-l are solved 
in the next section. 

Estimation using the exponential, normal, Weibull, gamma, 
and log normal distributions . -As an illustration of how to use 
these equations for an electrical part that experience indicates 
will follow the exponential distribution, consider example 1: 

Ex ample 1 : Testing of a particular tantalum capacitor showed 
that the failure density function was exponentially distributed. 
For the 100 specimens tested, it was found that the mean time 
between failures t was 1000 hours. 

(1) What is the hazard rate? 

(2) What is the failure rate at 100 hours and during the next 
10-hour interval? 

(3) What are the failure and reliability time functions. 

Solution 1: 

(1) Using the equations given in figure C-l for exponential 
distribution, the hazard rate is given by 


t 1000 hours/failure 


Letting h = ht in this equation gives 

t lim 1_ T R(t + At) - R(t) 1 

X '“A/-0 R{t) [ A/ J 

The term in brackets is recognized from the calculus to be 
the derivation of /?(/) with respect to time, and the negative 
of this derivation is equal top(f). Substituting these values 
gives 

1_ r mo] m Pit) 

X “ ~R(t) L dt J " R(t) 

As an example consider a jet airplane traveling from 
Cleveland to Miami. This distance is about 1500 miles and 
could be covered in about 2.5 hours. The average rate of speed 
would be 1500 miles divided by 2.5 hours, or 600 miles per 
hour. The instantaneous speed may have varied anywhere from 
0 to 700 miles per hour. The air speed at any given instant could 
be determined by reading the speed indicator in the cockpit. 
Replacing the distance continuum by failures, failure rate is 
p naingnm to average speed, 600 miles per hour in this example, 
and hazard rate is analogous to instantaneous speed, the speed 
indicator reading in this example. 


or 

X' = 1 x 10 -3 failure/hour 
(2) The failure rate is given by 



For this cast the time interval is given by 

h m t 2 - «i = 110 - 100 * 10 hours 
T • necessary reliability functions are given by 

e -h» =* f-no/iooo.,-0.11 « 0.896 

and 

= e - *00/1000 at 0.905 

Substituting these values gives 

X . — (\ - - j x 10~ 3 fa.lure/hour 

10 V 0.905 / 
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This is to be expected for the exponential case because die 
failure rate is constant with time and always equal to the 
hazard rate. 

(3) The failure and reliability time functions are given by 

f >(0 = - i - e -' / ' 000 


Therefore, using the data from table C-2, 

- 750 000 _ , 

t * — — — = 75 000 hours 

The unbiased standard deviation a is given by 


R(t ) = e~ ,nm 

As an illustration of how to use die equations given in figure 

C-l for mechanical parts subject to wear using the normal 
distribution, consider example 2: 

Example 2: A gimbal actuator is being used where friction, 
mechanical loading, and temperature are the principal failu re- 
causing stresses. Assume that tests to failure have been 

conducted on the mechanical parts, resulting in die data shown 

in table C-2. 

(1) What is the mean time between failures and die standard 
deviation? 

(2) What are the hazard rate at 85 300 hours and die failure 
rate during die next 10 300-hour interval? 

(3) What are the failure and reliability time functions? 

Solution 2: 

(1) The mean time between failures is given by 


- /-i 


" ( E/ ') 

/-I n 


n \2 — ■ 1/2 


The sum terms required for this calculation are gr r cn by 


E rj = 57 213 (10 3 hours) 2 (column 3, table C-2) 
/■ 1 


(j£pj * (750)J = 562 500 (10 3 hours) 2 
r m ^57 213 - 56 250 _ ^963 J 

(2) The hazard rate X' is given by 


10 300 hours 


t mean time between failures, hours 
t/ time to failure, hours 
« number of observations 

TABLE C-2.— TEST DATA FOR 
GIMBAL ACTUATORS 


Ordered Time to 




I 


60x10 s 

63 

66 

70 

75 

75 

60 

63 

85 

90 


Total 750x10 s 



Scaled ordinate at 85 300 hours 
Normal area from 85 300 hours to oo 

Let K, be the normal ordinate at 85 300 hours and Z, be the 
standardized normal variable, which is given by 

* _ f - (85 300 - 75 000) hours 

-M m — s — ■ ■ — ■ 



















Ex P on • n,,,,, jexpR/7] 


f-T) M 

I*-** 



ds " tp [ - 4( L 5T-) ] i, ]* 



ti-yf 

a 




Lognormal 


Distribution 


^4^11 





Poisson M f #xpN/f , 
"f I 


Hi p Nf g n-N f 
(n-ty)ltyr * 


R{N,) 


(fltympb-t/f] 



Y -2L-pJgn-j 

A*-w 


Binomial 






























P h09 


Remarks 


1 1t 


h - t g —t i 


Complex electrical 
systems 


Normal ordinate at t 


Mechanical systems 


Normal area r 1 




to “ 


a - scale parameter 
ft » shape parameter 
y - location parameter 


Mechanical or electrical systems. 

If a-tp-O.and y-0, reduces 
to exponential. If p - 3.5, approx- 
imates normal. 


Gamma ordinate at t 
Gamma area to** 


Same as Weibull parameters 
but may be harder to use. 


HP) - r f * 

o 

HP) - (P-DHp-i) 



Combined mechanical 
and electrical systems 

Log normal ordinate at t 
Log normal area f 1 to** 

Mechanical parts that fail due 
to some wearout mechanism 

X 

Remarks 

Not applicable 

Nf - number of failures 
One-shot devices 

Not applicable 

p • defectives 
g -effectives 
n - trials (sample size) 


Complex systems for 
.probability of Nf defects 



































Note that the denominator required to calculate X' is R(t t ), 
which is the normal area from 85 300 hours to oo. Existing 
tables for the normal area for Z, = 1.0 (ref. C-3) give the area 
from -oo to Z,, so that the unreliability Q(t t ) is given by 

Q(t t ) = 0.841 x (Area from -oo to Z,) 

Because Q(t,) + R(t,) - 1.000, 


(3) The constants required to write expressions forp(f) and 
R(t) are calculated as follows: 



1 


1 


• = 3.87 x 10 ~ 5 


o( 2v) in (1.03xl0 4 )x2.52 

2o 2 = 2 x (1.03X 10 4 ) 2 = 2.12X10 8 


i?(r,) = 1.000 - 0.841 =0.159 


Using the constants and substituting values gives 


and the hazard rate is given by 


2.35 X 10 " 4 failure/hour 
1.59x10-' 


= 1 .47 x 10 _3 failure/hour 


The failure rate is given by 



In this case h is given as 10 300 hours. The reliability at 95 600 
hours is given by 

R(t 2 ) = Normal area from 95 600 hours to oo 
Using the preceding procedure results in 
R(t 2 ) = 0.023 
Substituting values gives 

\ = (i Q- 023 ^ 8. 56 x IQ-' 

10 300 hours \ 0.159/ 1.03 x 10 4 

“ 8.31 x 10“ 5 failure/hour 


p(r) = 3.87X10 -5 < -<»-75xio*) , '2.i2xio» 

R(t) — 3.87 x 10 " 5 j ( e — fr-7 ' 5x,0 * )I/2 ‘ ,2x,aP dt 

As an illustration for the Weibull distribution, consider 
example 3: 

Example 3: A lot of 100 stepping motors was tested to see 
what their reliability functions were. A power supply furnished 
electrical pulses to each motor. Instrumentation recorded the 
number of continuous steps a motor made before it failed to 
step even though a pulse was provided. All testing was stopped 
at 1 x 10 6 steps. The step failure data are given in table C-3. 

(1) Calculate the frequency functions. 

(2) Plot the hazard rate function on log-log paper. 

(3) What conclusions can be drawn from this graph? 

Solution 3: Because there are 100 motors in this lot, die data 
give ordered plotting positions suitable for plotting on Weibull 
probability paper. Figure C-2 shows a plot of these data. From 
the shape of die data in figure C-2 it appears as though two 
straight lines are necessary to fit this failure density function. 
This means that different frequency functions exist at different 
times. These frequency functions are said to be separated by 
a partition parameter 5. 

From figure C-2 the Weibull scale, shape, and location 
parameters can be estimated by following the steps listed here: 

(1) Estimate the partition parameter 6. This estimate can 
be obtained direcdy from figure C-2. The two straight lines 


TABLE C-3. -WEIBULL DATA FOR STEPPING MOTORS 


Number of 
steps to 
failure 

Cumulative number 
of failures 

Median 

rank 

5-Percent 

rank 

95-Percent 

rani 

Problem 3 

Problem 9 

Scaled time to failure, t, 

0.2 xlO 3 

2 

I 

6./0 

0,51 

25.89 

.4 

4 

2 

16.23 

3.68 

39.42 

.9 

5 

3 

25.86 

8.73 

50.69 

4.0 

16 

4 

35.51 

15.00 

60.66 

10.0 

20 

5 

45.17 

22.24 

69.65 

18.0 

50 

6 

54.83 

30.35 

77.76 

30.9 

90 

7 

64.49 

39.34 

85.00 

50.0 

97 

8 

74.14 

49.30 

91.27 
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1 


-10 12 
log, (failure age) 

Figure C-2.— Weibull plot for stepping motors. 


■ that best fit the given data intersect at point f. Projecting this 
point down to the abscissa gives a failure age of 10 000 cycles 
for the partition parameter 5. 

(2) Estimate the location parameter y. This parameter is 
, used as a straightener for p(t). Because p(t - 0) is already 
j a straight line for both regions, it is clear that 71 « y t = 0. 
j In general, several tries at straightening may be required before 
i the one yielding a straight line for p{t - 7) is found, 
j (3) Estimate the shaping parameter 0. The intercept point 
i a for line b, drawn parallel to line c and passing through point 
! d, where ln(f - 7) = 1 is equal to 0. Thus, 0 ( = 0.75 and 
j ft* 150. 

| (4) Estimate the scale parameter a. At point e for line c, 

i 

i 

I In a = -In In 

! 1-0(0 

► j so that 

-J Tii 1 

! L 1 - 0(oJ 

i 

I 

| Therefore, 

i 

j o, » e 2 ' 75 « 15.7 



By using the parameters just estimated and the equations given 
in figure C-l for the Weibull distribution, the following failure 
frequency functions can be expressed: The partition limits on 
the number of steps c are 0 £ c £ 10 and c > 10. The 
frequency functions are given by 

/(c) =-(c- 7) fl -'e- (f -^ 

a 

Substituting values results in 

/,( C ) = 2l^ c 0.7S-l e -(c/l5.7)0’» 

or 

/, (c) = 0.047c " 0 2S e _c ° 75 15 7 for 0 s c S 10 

Similarly, 

f 2 (c) - 0.015c 0,1 e " c ' for c > 10 

The reliability functions are given by 

R(c) * 


100 





pjww T 1 w»^« *■*» 


,,sj.,^B*JMff.i Ulip U JJLL J 


Therefore, substituting values gives 

for 0 £ c £ 10 
and 

R 2 (t)=e- C '* m fore >10 
The failure rate functions are given by 

e -lc,-y,W‘i 

Therefore, substituting values gives 



Useful corollary equations are 

10* = y 
x = log Y 
10 °= 1 
and 

log 0.047 = log 4.7X10" 2 = log 4.7 + (-2)log 10 

= 2.672, or 8.672 - 10 


For c = 1 

log X,' = log 0.047 + (-0.25) log 1 
X,' = 0.047 




1 

h 




for 0s cis 10 


For c = 10 

log X,' = log 0.047 + (0.25) log 10 = 2.672 - 0.25 = 2.422 


and 


X,' = 0.0264 


*2 


1 

h 


1 


e -(c, )>»'«» 
. e -(C, )'•*'«» 


for c > 10 


In a similar manner solving for X 2 gives the data points 
shown in table C-4. These data are plotted in figure C-3. 


The hazard rate functions are given by 

X'=2(c-y/-' 

a 

Therefore, substituting values gives 

X/ » 0.047 c -025 for 0 S c s 10 
and 

X 2 ' = 0.015 c 05 fore >10 

(2) By using two-cycle log-log paper and the following 
calculation method, a graph of X' against c can be obtained: 

X,' = 0.047 c‘ 0 25 
Taking logarithms to the base 10 gives 


TABLE C-4 — HAZARD 
RATE DATA FOR 
STEPPING MOTORS 


Number of 
steps, 
c 

Failures 
per cycle, 
X' 

lxUP 

10 

10 

100 

0.047 

.026 

.015 

.150 


(3) Figure C-3 indicates that the hazard rate is decreasing 
by 0.25 during the first interval and increasing by 0.50 during 
the second interval for each logarithmic unit change of c. It 
appears that step motors, for first misses, jump from the 
“infant mortality” stage into the wearout stage without any 
transition period of random failures with a constant failure rate 
(ref. C-4). 

As an illustration of combined mechanical and electrical 
systems that follow foe gamma distribution, consider example 4: 
Example 4: Environmental testing of 10 electric rockets with 
associated power conditioning has resulted in the ordered time- 
to- failure data given in table C-5. 


log X,' - log 0.047 + (-0.25) log c 



•2 r- 


where 



4 6 8 10 20 40 60 IOOxIO 3 

Number of (tops or eyelet 


Figure C-3 — Hazard rate plot for stepping motors. 


i 

i 


TABLE C-5.— ELECTRIC ROCKET 
RELIABILITY DATA 


_ 

Ordered 

sample 

number 

Time to 
failure, 

hr 

Median 

rank 

Scaled 
time to 
failure 

Linear 

scale 

rank 

Scaled time to failure, 

1 

1 037.8 

6.70 

7.2 

5.0 

2 

1 814.4 

16.23 

12.6 

15.0 

3 

2 332.8 

25.86 

16.3 

25.0 

4 

3 124.8 

35.51 

21.7 

35.0 

5 

3 614.4 

45.71 

25.1 

45.0 

6 

4 579.2 

54.83 

31.8 

55.0 

7 

5 342.4 

64.49 

37.1 

65.0 

8 

6 292.8 

74.14 

43.7 

75.0 

9 

7 920.0 

83.77 

55.0 

85.0 

10 

II 404.8 

93.30 

79.2 

95.0 


tj i ,h scaled time to failure 

/go rough estimate of 80-percent failure time 

/, i ,h time to failure, hours 

Table C-5 gives /, for each ordered sample. 

(5) Plot on linear graph paper (10 x 10 to the inch) median 
rank against scaled time to failure /,. Figure C-5 shows the 
plotted data points for this problem. 

(6) These data points fit the gamma curve well with a 0 
es imate of 2.0; hence, it appears as though a two-parameter 
gat ima distribution is required with the location parameter y 
equal to zero. The nonzero location parameter case is covered 
in the literature (ref. C-5). 

(7) Overlay the linear axis (10 spaces to the inch) of a sheet 
of five-cycle semilog paper corresponding to a 0 of 2.0. Plot 
on this special graph paper the linear scale rank against time- 
to-failure data given in table C-5. 

(8) Fit a straight line through the plotted points. Figure C-6 
shows the plot for these data. Two additional straight lines 
are shown in this figure. Line 1 was obtained by plotting two 
known points (0.5,1) and (20,8) (ref. C-5). Line 2 has one 
point at (0.5,1) with a slope m. If line 1 were coincident with 
line 2, the 0 estimate would be sufficiently accurate. 

(9) Because the two lines are not coincident, a closer 
approximation for 0 is obtained by taking a new midpoint 
coordinate estimate of 6.8 from figure C-6. Using existing 
charts gives 0 = 2.25, which satisfies the slope criteria 
(ref. C-5). 

(10) For a shape parameter 0 of 2.25 a linear scale rank 
of 20 percent applies. Entering figure C-6 at this point on the 
ordinate gives a scale parameter a of 2400 hours. 


(1) What is the mean time between failures? 

| (2) Write the gamma failure and the reliability functions. 

\ (3) What is the hazard rate at 5000 hours? 

j (4) What is the failure rate at 5000 hours during the next 

j 1000-hour interval? 

| Solution 4: The essential steps for the graphical solution of 
j this problem are as follows (ref. C-5): 

(1) Obtain the median ranks for each ordered position; see 
table C-5. 

(2) Plot on linear graph paper (10 x 10 to the inch) median 
rank against time to failure for the range around 80-percent 
median rank. 

f (3) Fit a straight line to the plotted points. For a median 
rank of 80 read the corresponding time to failure / w in hours. 
1 Figure C-4 gives a /go of 7200 hours, 
j (4) The time-to-failure data are scaled by using the equation 

! 

i 



Figure C-4. -Electric rocket life. 
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With these graphical construction aids the solution to the 
problem is readily achieved: 

(1) The mean time between failures is given by 

t- a/3 = 2.4 x 10 3 hours x 2.25 = 5.4 x 10 3 hours 

(2) The gamma failure and reliability functions are given by 


Pit) = 


1 


it - y) 0 ~'e 


U-r V« 


Here 


PiU) = 41X10 7 (5x 103)1 ” *- 5x,0V2 - 4xl05 


Performing the indicated operations gives 


. . (4.21 X10 4 )X( 1.25 X10" 1 ) 

pM ‘ Srf -'"’“O ' 4 


! 

1 


« 

t 


l 

l 

i 

f 

! 


I 


1 

t 

\ 

j 


It has been shown that 7 = 0 ; the other constants are fainiinnvt 
as follows: 


a p = (2.4X10 3 ) 225 

Using logarithms, log a 6 * 2.25(log 2.4 + log 10 3 ); 
performing the indicated operations gives log a* = 7.61; 
hence, a tf = 4.25 xlO 7 . 

The second required constant is T(/3) = T(2.25). Using the 
identity T(jc + 1) = x /, then T(2.25) * T(1.25 + 1) = 1.251. 
Using Sterling’s formula, x\ = x x e~ x (2rx) m . T akin g 
logarithms gives 

leg x! m X log X + ( -X) log, + 0 [log 2 t + log x] 

* (* + 5) l ° s x ~ 0,434jt + 0,399 


log(1.25!) * 1.75 log 1.25 - 0.434 x 1.25 + 0.399 - 0.026 


Substituting and forming the product gives a*T(0) ■ (4.24 
x 10 7 ) x 1.06 * 4.5x 10 7 . Using these constants and substi- 
tuting values gives 


We can obtain /?(r,) either analytically by using this integral 
equation or graphically from figure C-6. Enter figure C-6 at 
a failure age of 5000 hours. Draw a vertical line to line 3. 
Project the intersection of/(r) and 5000 hours over to the 
linear scale rank (0.605). Using a previous identity. 


R(ti) * 1 -0.605 = 0.395 
Substituting values gives 


1.17X10” 4 

3.95x10”' 


= 2.71 x 10 ~ 4 failure/hour 


(4) Thife failure rate function at 5000 hours during the next 
1000-hour interval is given by 


*«,>J 

Following the procedure given previously and sub stituting 
values gives 


Rih) = 1 - 0.710 » 0.290 


p(t) m *1.25 - -1/2.4* I0 J 

PK) 4.5 xio 7 * 


and 


and 


*(/)■- ! -[ ,1.23 -r/2.4xiO> A 

4.5xl0 7j » e " 

(3) The hazard rate flutction at 5000 hours is given by 
X' 

* 0 .) 



2.65 x 10 ” 4 failure/hour 


As an illustration of mechanical parts, consider example 5: 
Example J: A cable used as guy supports for sail experiments 
in wind tunnel testing exhibited the time-to-failure performance 
data given in table C-6. 

(1) Write the failure and reliability functions. 

(2) - What is the hazard rate at 5715 hours? 

(3) What is the failure rate during the next 3000 hours? 
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TABLE C-6,— TEST DATA FOR GUY SUPPORTS 


Ordered 

sample 

number 

Time to 
failure, 

'/• 

hr 

Median 

rank 

5-Percent 

rank 

95-Percent 

rank 

1 

1 100 

6.7 

0.5 

25.9 

2 

1 890 

16.2 

3.7 

39.4 

3 

2 920 

25.9 

8.7 

50.7 

4 

4 100 

35.5 

15.0 

60.7 

5 

5 715 

45.2 

22.2 

69.7 

6 

8 720 

54.8 

30.3 

77.8 

7 

12 000 

64.5 

39.3 

85.0 

8 

17 500 

74.1 

49.3 

91.3 

9 

23 900 

83.3 

60.6 

96.3 

10 

46 020 

93.3 

74.1 

99.5 


Solution 5: 

(1) The essentia] steps for solving this problem are given 
here: 

(a) Obtain the median rank for each ordered position, see 
table C- 6 . 

(b) Plot median rank against time to failure on log-normal 
probability graph paper (probability times two log cycles), as 
shown in figure C-7. 

(c) If a straight line can be fitted to these plotted points, 
the time-to-failure function is log normal. 

(d) JThe mean time between failures is calculated by 
t' = ln(f), where t — 6970 hours as shown in figure C-7 for 
a median rank of 50 percent; hence t ' = 8.84. 

(e) The standard deviation is given by 




80x10 a 



flit,) Wz) 

Mill 


.98 .90 .70 .50 .30 .10 

1-Rank 


.02 


Figure C-7.— Guy support life. 


o r 


In t(j - In tj 
3 


where t(j ■ 49 500 hours and t[ = 1020 hours as shown in 
figure C-7 for a median rank and a 1 - rank of 93.3 percent; 
hence, o r « (10.81 - 6.93)/3 » 1.28. 

With these constants the expressions for p(t) and R(t) are 
written as 


Pit) 


3.21 X IQ' 1 

f 


0 - O' -S.84F/3.28X 10 


t' ~t' _ 8.66-8.84 
a,- 1.28 


-0.143 


From the normal-curve ordinate tables 


and 


Y{ - 0.395 


NYj 10 x 0.395 
<V " 1.28 


3.09 failures 


Substituting values gives 


and 

rt(r) » 3 . 21 x 10 -' J e -U'-tMWMxio A 

(2) The log-normal ordinate required for X' cm be calcu- 
lated by using the standardized normal variable table as in 
example 2. The log-normal standardized variable is given by 


V 3 QQ 

p(t') » — * — — j — —5 “ 5.40x 10 -4 failure/hour 
t 5.715 x 10 s 

The log-normal area from t' to infinity can be obtained 
directly from figure C-7 by using the 1 - rank scale. Enter 
the time-to-failure ordinate at 5715 hours; project over to the 
log-normal file ftinction /(f) and down to the 1 - rank abscissa 
value of 0.638. Therefore, the hazard rate X' at 5715 hours 
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is given by 


5.40x10' 


6.38x10' 


• = 8.46x 10“ 4 failure/hour 


(3) The failure rate during the next 3000 hours is calculated 
by knowing that /?(*,) = -0.638 at a time to failure of 5715 
hours and obtaining R(t 2 ) = 0.437 from figure C-7 at 8715 
hours. Therefore, the failure rate is given by 


(* “SSO = 105xI0 " 4 failure/hour 


Determination of confidence limits .— In the preceding 
sections statistical estimates of various parameters have been 
made. Here we determine the methods for defining the con- 
fidence to be placed n some of these estimates. In example 1 
tantalum capacitors with a one-parameter exponential dis- 
tribution were studied. For an exponentially distributed 
population, additional estimates follow the chi-squared distri- 
bution. As an illustration of how to determine confidence limits 
for an exponentially distributed estimate, consider example 6. 

Example 6: One hundred tantalum capacitors were tested 
for 15 000 hours, during which time 15 parts failed. 

(1) What is the mean time between failures? 

(2) What are the upper and lower confidence limits at 
98-percent confidence level? 


Solution 6 : 

(1) The mean time between failures is given by 


r T 15 000 hours 

1 “ ~ " , .. “ 1000 hours/failure 

r 15 failures 


(2) The upper and lower confidence limits at some 
confidence level are given by 


X[l-(a/2));2r 


/ 2f ~ 
L ■ — t 

x my. Jr 


upper confidence limit, hours 
lower confidence limit, hours 


total observed operating time, hours 


X percentage points of chi-squared distribution 

r number of failures 

1 - a/2, probabilities that 7 will be in calculated 

a/2 interval 


For the 98-percent confidence level required by this problem 


? = 0.01 
2 


1 -- = 0.99 
2 


2r = 30 


Therefore, the chi-squared distribution values are given by 
(available from many existing tables) 


X0.0l;30 — 50-9 


X0.99;30 = 14.9 


Substituting values gives 


„ 30x 1000 
U " — r— — • 2013 hours 
14.9 


30 x 1000 


= 589 hours 


Thus, it is known with 98-percent confidence that the limits 
of the time 7 lie between approximately 590 and 2010 hours. 

Determining the percentage values for the chi-squared 
distribution for values of r greater than 30 may also be useful. 
It has been shown that when r s» 30, 


I2(2r) - 1] 


- til* 


where Z is the area under the normal curve at the specified 
confidence level. Example 7 illustrates how this equation is 
used for confidence interval calculations. 

Example 7: The tantalum capacitors of example 6 have been 
operated for 5000 more hours; five additional units have failed. 
What are the confidence limits on rat the 98-percent confidence 
level for this additional testing? 

Solution 7: For the areas under the normal curve from - oo 
to Z equal to 0.98 and 0.02, existing area tables give 
Z ■ ±2.06 and r - 15 + 5 ■ 20 total failures, with 2r ■ 40. 





w 
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Substituting values gives 

(0,0 1/2 = (2 x 40 - l) l/2 ± 2.06 
Xo.0l;40 = 59.7, X 0.99:40 = 23.4 


Hence, 


U- 


40xl0 3 


23.4 


= 1709 hours 


L = 


40xl0 3 


59.7 


! 670 hours 


Thus, it can be said, with 98-percent confidence that t lies 
between approximately 670 and 1710 hours; as the test time 
increases, die estimated-parameter confidence interval decreases. 

In example 2 gimbal actuators that exhibited normally 
distributed time-to-failure data were analyzed. For a normally 
distributed population, additional mean estimates will also be 
normal. As an illustration of how to determine confidence 
intervals for normal estimates, consider example 8. 

Example 8: Twenty-five gimbal actuators have been tested. 
The mean time between failures has been calculated to be 
75 000 hours with a standard deviation of 10 300 hours (see 
example 2). What are the upper and lower confidence limits 
at a 90- percent confidence level? 

Solution 8: The upper and lower confidence limits are given by 


U -t + K. 


’ar/2 i/2 

n lu 


L = t 


K ^ 

A o /2 | /2 


where 

t 

R«n 

o 

n 

1 - a 


mean time between failures, hours 
standardized normal variable 
unbiased standard deviation 
number of samples 

probability that t will be in calculated interval 


For this problem 


1 -a-0.90 


a -0.10 


£-0.05 

2 


and from existing tables for the area under the normal curve 
K„n “ 1-64. Substituting values gives 
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U - 75 000 + -i^ 300 = 78 400 hours 


25 


1/2 


and 


r ,*««« 1 64X 10 300 

L = 15 000 71 600 hours 


25 


1/2 


This means that 90_percent of the time the mean-time-between- 
failures estimate t for 25 gimbal actuators, rather than the 
original 10, will be between 71 600 and 78 400 hours. Note 
that the sample size n has been increased to use this technique. 
This reflects the usual user pressure to learn as much as 
possible with the least amount of testing. Try to keep n 2 25 
in estimating normal parameters with this technique. Ifn < 25, 
use Student’s t distribution (ref. C-6). To determine the effects 
of reducing sample size on confidence intervals, rework 
example 2 for the smaller sample size of 10, using Student’s 
/ distribution. The upper and lower confidence limits are 
given by 


U- 


' t + tan ~2 
n u 


and 


L ~ t~ tali — 

n 1 


where 

t a Q Student’s t variable 
s standard deviation 


For this problem, r = n - 1 = 9, a = 0. 10, and t al2 from 
existing tables is 1.83. The standard deviation is given by 


/ 57 213 - $6 230 \ 1/2 

V 10 / 


9820 


Substituting values gives 


(/ » 75 000 + 1-83 * 9820 * 80 700 hours 


10 


1/2 


and 


L - 75 000 + >;83 X 9820 - 69 300 hours 


10 ,/J 







Comparing this time interval with that calculated for a sample 
size of 25 shows that the smaller sample gives a larger interval 
of uncertainty. 

In example 3 stepping motors that exhibited Weibull- 
distributed time-to-failure data were studied. As a graphical 
illustration of how to determine confidence intervals for a 
Weibull-distributed estimate, consider example 9. 

Example 9: Another group of stepping motors has been step 
tested as previously explained in example 3. The Weibull plot 
of percent failures for a given failure age is the same as that 
given in figure C-2. During this testing, however, only eight 
failures have occurred. What is the 90-percent confidence band 
on the reliability estimate at 4000 cycles? 

Solution 9: The data needed for graphical construction of 
the confidence lines on the Weibull plot are given in table C-3. 
The steps necessary to construct the confidence lines in figure 
C-2 are as follows: 

(1) Enter the percent failure axis with the first 5-percent 
rank value hitting/(f); for failure 2 the 5-percent rank is 3.68. 

(2) Draw a horizontal line that intersects /(/) at point 1. 

(3) Draw a vertical line to cross the corresponding median 
rank; for failure 2 the median rank is 16.23. 

(4) Draw a horizontal line at die median rank, 16.23, for 
failure 2. The intersection point of the line for step (3) with 
this line is one point on the 95-percent confidence line. 

(5) Repeat steps (1) to (4) until the desired cycle life is 
covered, 4000 cycles in this case. 

(6) The 5-percent confidence line is obtained in a similar 
manner. Enter the percent failure axis with the 95-percent 
failure rank; 25.89 for failure 1. 

(7) Draw a horizontal line that intersects /(r) at point 3. 

(8) Draw a vertical line to cross the corresponding median 
rank; 6.70 for failure 1. 

(9) Draw a horizontal line at the median rank, 6.70, for 
failure 1 . The intersection point of these two lines is one point 
on the 5-percent confidence line. 

(10) Repeat steps (6) to (9) until the desired cycle life is 
covered. 

A 90-percent confidence interval for /(/) at 4000 cycles is, 
from figure C-2, 1.2 percent to 37.5 percent. Hence, a 
90-percent confidence interval for R(t) at 4000 cycles is 0.998 
to 0.625. 

In example 5 guy supports that exhibited log-normally- 
distributed time-to-failure data were analyzed. As a final 
graphical illustration of how to dem mine confidence intervals 
for a log-rormally-distributed estimate, consider example 10. 

Example 10: It has been shown that the guy supports of 
example 5 exhibited a reliability of 0.638 at a time to failure 
of 5715 hours. Consider now foe procedure for determining 
the confidence band on this log-normal estimate. The data 
needed for the graphical construction of the 90-percent 
confidence lines on foe log-normal graph of figure C-7 are 
also given in table C-6. 

Solution 10: The steps necessary to graphically construct 
the confidence lines in figure C-7 are as follows: 


(1) Enter the rank axis with the first 5-percent rank value 
hitting/(f) , the log-normal life function shown in figure C-7; 
for ordered sample 3 the 5-percent rank is 8.7. 

(2) Draw a vertical line to intersect f(t) at point 1 as shown 
in figure C-7. 

(3) Draw a horizontal line to cross the corresponding 
median rank; for ordered sample 3 the median rank is 25.9. 

(4) The intersection point (point 2 in fig. C-7) of step (3) 
and the median-rank line is one point on the 95-perrent 
confidence line. 

(5) Repeat steps (1) to (4) until the desired time to failure 
is covered; 5715 hours in this case. 

(6) The 5-percent confidence line is obtained in a similar 
manner. Enter the rank axis with the 95-percent-failure rank, 
25.9, for ordered sample 1. 

(7) Draw a vertical line intersecting/!/) at point 3. 

(8) Draw a horizontal line to cross the corresponding 
median rank; for orde-rd sample 1 die median rank is 6.7. 

(9) The intersection point (point 4 in fig. C-7) of these two 
lines is one point on the 5-percent confidence line. 

(10) Repeat steps (6) to (9) until the desired time to future 
is covered. 

At 5715 hours the 90-percent confidence interval for /(/) is, 
from figure C-7, 19.7 to 69.4 percent. Hence, a 90-percent 
confidence interval for /?(/) at 5715 hours is 0.803 to 0.306. 
Incidentally, this graphical procedure for finding confidence 
intervals is completely general and can be used on other types 
of life test diagrams. 

Estimation using the Poisson and binomial events.— The 
binomial and Poisson distributions are discrete functions of 
the number of failures tythat occur rather than of the time /. 

The Poisson distribution (fig. C-l) is a discrete function 
of the number of failures. When this distribution applies, it 
is of interest to determine the probabilities associated with a 
specified number of failures in the time continuum. As an 
illustration for a complex electrical component that follows 
die Poisson distribution, consider example 1 1 . 

Example 11: Ten space-power speed controllers were tested 
during the rotating solar dynamic development program. The 
time-to-failure test data are given in table C-7. 

(1) Write the Poisson failure density and reliability 
functions. 

(2) What is the probability of five failures in 10 000 hours? 

(3) What is the probability that 6, 7, 8, 9, or 10 failures 
will occur? What is the reliability after the fifth future? 

Solution 11: 

(1) Reducing foe data given in table C-7 gives foe mean 
time between failures as 


to 

Li, 

im | 


8.59 x 10 4 


8.59 xl0 } hours/failure 


10 


TABLE C-7.-PCHSSON DATA 
FOR SPEED CONTROLLER 


/ — 1 -1 

UluCIOQ 

sample 

Time lo 
feUtire, 

¥ 

hr 

1 

3 520.0 

2 

4 671.2 

3 

6 729.3 

4 

7010.0 

5 

8 510.2 

6 

9 250.1 

7 

10910.0 

8 

11220.5 

9 

11 815.6 

10 

12226.4 

Total 

85 866.3 


Hence, the Poisson failure den* v function is given by 


p{N f ) 


( ! 

\8.59xl0 3 


■r . 


j/S. 59x10* 


A?/ 


The reliability function is given by 


10 

J»(Ar r ) * E 


f ' Y 

\8.59x 10y 


f/S. 59x10* 


>»' ■>! 


to 


R(N f ) = L 

j-6 


0.314(1. 16)> 

jl 


Calculating each term and summing gives 
*(6)«0.f-:i3 

The binomial distribution is given in figure C-l as 
distribution 7. Considerable work has been done to develop 
the techniques suitable for using this powerful tool (refs. C-l 
and C-3) As an illustration consider a pyrotechnic part 
described in example 12. 

Example 12: A suspicious lot of explosive bolts is estimated 
to be 1$ percent defective due to improper loading deputy as 
observed by neutron radiography. 

(1) Calculate the probability of one defective unk appearing 
in a flight quantity of four. 

(2) Mot die resulting histogram. 

(3) What is the reliability after the first defect? 

Not many failure density data are available, but past experience 
with pyrotechnic devices has shown that the binomial 
distribution applies. From the given data the per-unit number 
of effectives q is 0.85, the per-unit number of defectives p 
is 0.15, the sample size n is 4, and the possible number of 
failures N f is 0, 1, 2, 3, or 4. The frequency functions 
corresponding to these constants are given by 


p(N f ) - 


4! 


(4 - N f )lNf. 


N. 4-N. 

P '<! ‘ 


(2) To calculate the probability of live failures in 10 000 
hours, use the ratio 


1.0x10 

8.59x10* 


The probability of five failures in 10 000 hours is given by 


Pi 5) 


5! 


2.09 x 0 314 
1.2x10* 


5.47X10" 3 


One easy method of calculating the term (1.16) 5 is as follows: 
togO.16) 3 >5 log 1.16 “5(0.148) *0.740 
(1.16) 5 -2.09 

(3) The reliability from the 5th to the 10th failure is the sum 
of the remaining terms in the Poisson expansion. The Poisson 
expansion in sum form is given by 


and 


4 

Wf) = E — i! — piq'-j 

j-N, (4 -m KH 

One simple method for obtaining the binomial expansion 
coefficients is to make use of Pascal’s triangle. Pascal found 
that there was symmetry to the coefficient development and 
explained it as shown in table C-8. Pascal’s triangle (dashed 
lines) is shown in the last column. The lower number in the I 


~l Afll £ C— i. • -BINOMIAL 
EXPANSION COEFFICIENTS 


Sample 

tire. 

A 

Possible 
number of 
fattares 

Binomial 

expansion 

coefficients 

1 

2 

i 

2 

3 

1 2 1 

3 

4 


4 

5 

1 4\6/4 1 




dashed triangle is obtained by adding the two upper numbers 
(i.e., 3 + 3 = 6). 

Using these constants and expanding gives p(N f ) as 

P(Nj) = q* + 4 q l p + 6 q 2 p 2 + 4 qp 3 + p* 

The probability of one defective unit appearing in a flight 
quantity of four is given by the second term in the expansion; 
hence. 


4 q 3 p - 4(0.85) 3 (0. 15) = 0.37 

The resulting histogram for this distribution is shown in flgure 

C-8. The probability that 2, 3, or 4 defects will occur, as the 
reliability after the first defect, is the sum of the remaining 
terms in the binomial expansion. This probability can be 
calculated by using the equation for R(Nj). However, it is 
simpler to use the histogram graph and sum the probabilities 
over Nj from 2 to 4; hence, 

1?(2) = 0.096 + 0.011 +0.0011 =0.108 

These explosive bolts in their present form are not suitable 
for use on any spacecraft because the probability of zero 
defects is only 0.522, much below the usually desired 0.999 
for pyrotechnic spacecraft devices. 

Determination of confidence limits .— When an estimate is 
made from discrete distributions, it is expected that additional 
estimates of the same parameter will be close to the original 

estimate. It is desirable to be able to determine upper and lower 

confidence limits at some stated confidence level for discrete 
distribution estimates just as is done for continuous functions 
of time. The analytical procedure for determining these inter- 
vals is simp ified by using specially prepared tables and graphs. 
Useful tables for the binomial distribution are given in the 
literature (ref. C-3). 

As an example of how confidence intervals can be 
for Poisson estimates, consider ptoblem 13. 

Problem 13: The Poisson estimate of reliability from the 
5th to the 10th failure for speed controllers was found to be 



Number of (alim, N t 
Fi«ure C-8.— Expletive bolts histogram. 


a 

4 


0.0013 in a previous problem. What are the upper and lower 
confidence limits on this estimate at a 95 -percent 
level? 

The variation in / can be found by using figure C-9. Enter 
figure C-9 on the 5-percent a line at foe left-hand end of 
foe 5 interval. Here T/J, = 10.5; then /, = = 

8.57x 10 4 /10.5 » 8160 hours. Using the left-hand end of foe 
4 interval gives T/7 2 = 9.25; then t 2 = 8.57xl0 4 /9.25 = 
9530 hours. One simple method for finding f(5) « m use fig ure 
C-10(ref. C-5). The /A ratios of interest are 1.22, 1.16, and 
1.05, respectively. For these ratios with 5 the values of 
/( 5) from figure C-10 are 0.997, 0.P987, and 0.99992, 
respectively. Because the sum of the last five terms is desired 
R(5) is 0.003, 0.0013 and 0.0008. respectively. This meani 
that the probability of the 5th to the 10A failure of a speed 
control occurring is in the interval 0.0008 to 0.003 at a 
confidence level of 95 percent. 

As an illustration of how confidence intervals can be 
obtained for a binomial distribution, consider example 14. 

Example 14: The probability of one defective unit apji^fjii^ 
in a flight quantity of four explosive bolts has been 
to be 0.37. What are foe upper and lower confidence limits 
on this estimate at a 90-percent confidence level? 

If the sample size is n, die number of defectives is r, and 
the confidence level is y, this example has the following 
constraints: » = 4, r = 1, and <y = 90 percent. Using these 
constraints, the upper U and lower L confidence limits can 
be obtained directly from existing tables as U = 0.680 and 
L = 0.026. This means that with a 90-percent confidence level 
the probability of one defective bolt appearing in a flight 
quantity of four is in foe interval from 0.026 to 0.680. 

Sampling 

Purpose of sampling. —Sampling is a statistical method used 
when it is not practical to study foe whole population. There 
are usually five basic reasons why sampling is necessary: 

(1) Economy— It usuai'y costs less money to study a sample 
of an item than the whole population. 

(2) Timeliness— A sample can be studied in less time than 
the whole population, giving prompt results. 

(3) Destructive nature of a test-Some tests require that the 
end item must be used up to demonstrate performance, leaving 
nothing to use. 

(4) Accuracy— A sample survey accomplished by well- 
tnuned researchers usually will result in accurate and valid 
decisions. 

(5) Infinite population— In many analytical stndfrg an 
infinite population is available. If any information is to be used 
for decision making, it must be based on a samp le 

Choosing a sample. — Good judgment must be used in 
choosing a sample. Subjective methods of choosing samples 
frequently result in bias. Bias is an expression, either conscious 
or subconscious, of the selector’s preferences. Bias can be held 
to a minimum by using a nonsubjective method developed just 
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Figure C-10.— Poisson unreliability sum. 


for this purpose. Several nonsubjective sampling procedures 
are described here: 


(1) Random sampling— Each item in die population has an 
equal and independent chance of being selected as a sample. 
A random-digits table, see figure C-l 1, has been developed 
to facilitate drawing random samples. This table has been 
construe**! to make the 10 digits from 0 to ? equally likely 
to appear at any location in the table. Adjacent columns of 
numbers can be combined to get various-sized random 
numbers. 

(2) Stratified sampling— Similar items in a population are 
grouped or stratified, and a random sample is selected from 
each group. 

(3) Ouster sampling— Items in a population are partitioned 
into clusters, and a random sample is selected from each 
cluster. 

(4) Double sampling— A random sample is selected; then, 
depending on what is learned, some action is taken or a second 


sample is drawn. After the second random sample is drawn, 
action is taken on the basis of data obtained from die combi- 
nation of both samples. 

(5) Sequential sampling— Random samples are selected and 
studied one at a time. A decision on whether to take action 
or to continue sampling is made after each observation on the 
basis of all data available at that selection. 

As an illustration of when to use various sampling methods 
consider example IS. 

Example 15: Describe how a sample should be selected for 
three cases: 


(1) Invoices numbered from 6721 to 8966 consecutively. 
A random sampling procedure could be used in this case based 
on the four-digit table given in figure C-l 1 . Using the given 
invoice numbers , start at die top of die left column and proceed 
down each column selecting random digits until the desired 
sample size is obtained. Disregard numbers outside the range 
of interest. 








6433 

3465 

9601 

2364 

7304 

3764 

0251 

2031 

6398 

0911 

5052 

9225 

3100 

4598 

9WO 

9360 

6796 

7071 

7336 

1660 

7506 

5000 

4255 

5764 

3609 

7218 

6854 

4403 

2976 

1072 

6488 

9263 

0357 

5372 

6570 


4177 

8438 

5820 

7721 


2582 

7348 

9189 


9292 

5460 

3139 

0919 

1374 

3930 

5023 

3984 

7916 

0065 

7285 

1041 

9974 

8254 

4451 

0222 

1020 

8237 

6894 

9837 

1368 

5939 

5911 

4263 

4381 

2292 

0300 

7756 

3493 

9351 

COOO 

8251 

0092 

4892 

6287 

3804 


0620 

5774 

0141 

1430 

4580 

6385 

4201 

7613 

1904 

0324 

3045 


9757 

4257 

0480 

2094 

1913 

8825 


1460 

3821 

1377 

9505 

8160 

9045 

0578 

1535 

7490 

8151 

3433 

4642 


6557 

1411 

4212 

6309 

3020 


2005 0215 


8718 

6203 

8093 

6780 

9129 

4932 

1495 

4755 

2206 

4428 

9530 

6300 

0385 

8393 

7565 

0336 

4207 

2089 

7484 

9520 


6829 

9191 

7490 

7113 

5465 

4940 

5451 

9638 

4934 

8316 

6793 

4451 

6023 

7871 

8119 

7386 

5509 

0339 

6184 


6606 

6216 

3467 

3146 

7144 

7170 

2172 

1610 

3941 

3365 

6365 

7260 

5307 

4638 

7766 

2623 

4943 

9000 

5344 

2370 

1892 

5325 

5011 

5412 

3099 

6648 

4630 

8251 

6946 

8183 

7709 

7769 

4313 

2811 

9490 

1966 

9891 

2054 


7143 

2148 

7971 

4815 

8073 

5831 

6876 

7491 

0284 


9152 


7310 

1383 

2691 

8418 

3377 

2384 

9423 

4673 

0714 

2687 

8245 

5784 

0452 

4869 

1887 

6365 

4514 

2652 

7126 

7385 

9022 

3099 

3024 

1744 

9050 

9115 

1149 

9024 

0968 

1853 


9158 

1221 

0811 

9732 

8476 

4668 

4347 

3255 

5817 

0566 

5073 

7625 

0786 

7398 

5023 

6422 

9143 

6129 

1856 

3039 

7249 

8720 

6199 


5114 


0544 

4179 

0942 

6207 

9039 

3236 

8041 

3606 

8243 

2306 

4454 

4202 

3429 

1213 

3675 

8640 


8309 

3447 

1896 

9388 

4288 

4014 

1630 

5047 

5416 

7512 

2701 

9790 

0227 

5374 

4683 

0176 

0451 

7953 

6023 

5053 

0009 

4183 

6415 


7218 

4841 

9194 

7748 

5564 

2468 

4920 

7083 

3475 

7785 

7062 

5791 

2440 

3601 


9491 

7942 

0504 

7705 

6661 

3979 

1514 

3614 

4629 

8471 

2332 

8547 

0102 

5074 

8047 

0651 

4436 

3670 

7855 

1960 

4602 

6347 

8086 

8671 

9148 

9803 

7382 

3528 

6676 

4488 

6667 

2574 

3523 

4330 

8319 

5269 

4622 

2543 

4000 

5606 


8063 

9971 

4606 

4532 

1285 

1116 

9985 

UflO 

ww 

6773 

6166 

0922 

7343 

5745 

8018 

1887 

8673 

8413 



COM 

6£79 

4227 

1112 

5170 

4008 

4381 

5572 

2145 

7665 

4396 

1351 

5329 

5230 

9644 

7278 

2972 

5941 

8415 

7863 

5148 

7218 


Figure C-l 1. — Random digits table. 


(2) Printed circuit assemblies to compare the effectiveness 
of different soldering methods. If boards are all of the 
type, a cluster sampling procedure could be used here. Group 
the boards by soldering methods; select x joints from each 
cluster to compare the effectiveness of different soldering 
methods. 

(3) Residual gases in a vacuum vessel to determine the 
partial pressure of gases at various tank locations. A stratified 
sampling procedure could be used in this case. Stratify the 
tank near existing feedthroughs into Jt sections; an appropriate 
mass run could be taken from each section at various ionizer 
distances from the tank walls. Analysis would tell how the 
partial pressures varied with ionizer depth at the feedthrough 
locations. 

Sample sizt.-A completely general equation for dete rminin g 
sample size n is given by 


QU) = 1 -/?</,) =3 

n 


where 

Nj desired number of time-to- failure points 
n sample size 
t, test truncation time 

This equation can be used with any of the reliability functions 
given in figure C- 1. 

As an illustration of how these equations can be applied tp 
electrical parts, consider example 16, which is derived fiom 
example 1. 

Example 16: Tantalum capacitors with a Mure rate of 
lxlO" 3 failure/hour are to be tested to Mure. In a 
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1000-hour test what sample size should be used to get 25 time- 
to-failure data points? 

Solution 16: The truncated exponential reliability function 
is given by 

R(t,) = e~' ,/l000 — 0.37 

Solving die general sample size equation for n and substituting 
values gives 


1 - R(t,) 0.63 

Rounding off to the nearest whole unit gives n = 40 pieces. 
This means that 40 capacitors tested for 1000 hours should 
give 24 time-to-Muie data points. 

Accelerated Life Testing 

Life testing to define the time duration during which a device 
performs satisfactorily is an important measurement in relia- 
bility testing because it is a measure of the reliability of a 
device. The life that a device will exhibit is very much 
dependent on the stresses it is subjected to. The same devices 
in field application are frequently subjected to different stresses 
at varying times. It should be recognized then that life testing 
involves the following environmental factors: 

(1) The use stresses may influence the device’s life and 
failure rate functions. 

(2) The field stresses could be multidimensional. 

(3) In the multidimensional stress space there is an inter- 
dependence among the stress effects. 

(4) Because most devices operate over a range in a multi- 
dimensional stress space, life performance may vary. 

Testing objects to failure ut.der multidimensional stress 
conditions is usually not practical. Even if it were, if the system 
were properly designed, the waiting time to failure would be 
quite long and therefore unrealistic. It has been shown that 
time-to-failure data are important to reliability testing, and now 
they appear difficult to obtain. These are some of the reasons 
why many are turning to accelerated life testing, such as 
compressed-time testing, advanced-stress testing, or optimum 
life estimates: 

(1) Compressed-time testing— If a device is expected to 
operate once in a given time period on a repeated cycle, life 
testing of this device may be accelerated by reducing the 
operating time cycle. The multidimensional stress condition 
need not be changed. The stresses are being applied at a faster 
rate to accelerate device deterioration. Care should be taken 
not to accelerate the repetition rate beyond conditions that 
allow the device to operate in accordance with specifications. 
Such acceleration would move the device into a multidimen- 
sional stress region that does not exist in field conditions and 


would yield biased information. As an illustration of compressed 
time testing, consider example 17. 

Example 17: The stepping motor in example 3 was being 
pulsed for life testing. How could this life test be accelerated? 

The power supply providing the stepping pulses may have 
been stepping at die rate of one pulse per 10 seconds, resulting 
in a test time of 10 7 seconds. These motors had a frequency 
response allowing for 10 pulses per second. Increasing the 
pulse stepping rate up to the frequency response limit yields 
comparable time-to-failure data in 10 s seconds, a savings in 
time of two orders of magnitude. 

(2) Advanced-stress testing— If a device is expected to 
operate in a defined multidimensional stress region, life testing 
of this device may be accelerated by changing the multi- 
dimensional stress boundary. Usually the changes will be 
toward increased stresses because this tends to reduce time 
to failure. There are two basic reasons why advanced stress 
testing is used: 

(a) To save time 

(b) To see how a device performs under these stress 
conditions 

Care should be exercised in changing stress boundaries to be 
sure that unrealistic conditions leading to wrong conclusions 
are not imposed on the device. A thorough study of the failure 
mechanisms should be made to ensure that proposed changes 
will not introduce new mechanisms that are not normally 
encountered. If an item has a certain Mure density distribution 
in the rated multidimensional stress region, changing the stress 
boundaries should not change the failure density distribution. 
Some guidelines for planning advanced-stress tests are as 
follows: 

(a) Define the multidimensional stress region for an item; 
nominal values should be centrally located. 

(b) Study the failure mechanisms applicable to this item. 

(c) On die basis of guidelines (a) and (b) decide which 
stresses can be advanced without changing the failure 
mechanisms. 

(d) Specify multiple stress tests to establish trends; one point 
should be on the outer surface of the multidimensional region. 

(e) Be sure that the specimen size at each stress level is 
adequate to identify the failure density function and that it has 
not changed from level to level. 

(f) Pay attention to the types of failures that occur at various 
stress levels to be sure that new failure mechanisms are not 
being introduced. 

(g) Decide whether new techniques being developed for 
advanced-stress testing apply to this item. Several popular 
techniques are described here: 

(i) Sensitivity testing— Test an item at the boundary 
stress for a given time. If Mure occurs, reduce stress by a 
fixed amount and retest for the same time. If no failure occurs, 
increase stress by a fixed amount and retest for the same time. 
Repeat this process until 23 failures occur. This technique is 
used to define endurance limits for items. 
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(ii) Least-of-JV testing— Cluster items in groups, subject 
each cluster to a specified stress for a given time. Stop at the 
first failure at each stress level. Examine failed items to ensure 
conformance to expected failure mechanisms. 

(iii) Progressive-stress testing— Test an item by starting 
at the central region in stress space and linearly accelerating 
stress with time until failure occurs. Observe both die failure 
stress level and die rate of increasing stress. Vary the rate of 
increasing stress and observe its effect on the failure stress 
magnitude. Examine failed items to ensure conformance to 
expected failure mechanics. 

As an illustration of advanced-stress testing, consider 
example 18. 

Example 18: A power-conditioning supply was being life 
tested at nominal conditions with an associated electric rocket. 
The nominal electrical, thermal, vibration, shock, and vacuum 
stresses resulted in fairly long waiting periods to failure. 
Changing the multidimensional stress conditions by a factor 
of 1.25 to 2, which is usually done during development testing, 
tended to identify design deficiencies with shorter waiting 
periods without affecting the failure mechanism. 

(3) Optimum life estimate— One remaining calculation for 
nonreplacement failure or time-truncated life test is the 
optimum estimate of mean time between Mures t. It has been 
shown (ref. C-l) that t given by the time sum divided by the 
number of Mures should be modified by a censorship factor 
and a truncation time factor. The censorship factor K is caused 
by wearout Mures, operator error, manufacturing errors, etc. 
The correction equation for t is given by (ref. C-l) 




$ 

L ti+ (n- N f )t, 


N f -K 


where 


N f number of failures 


K censorship factor 


As an illustration consider example 19. 

Example 19: The tantalum capacitor tested in example 1 
could have been stopped when 10 capacitors (580 part-hours) 
out of 100 had failed at a testing time of 100 hours. What is 
an optimistic value for 7 ? 

Solution 19: Inspection of die 10 failed capacitors showed 
that two units Med owing to manufacturing errors. Therefore, 
Nf m 10, K ■* 2, n ■ 100 capacitors, t, * 100 hours, and the 
sum of tj ■ 580 hours. Substituting these values into the t 
correction equation gives 


- 580 + 000- 10)100 


10-2 


■ 1197 hours 
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This is an optimistic estimate for the mean time between 
failures, but it certainly is fair and reasonable to make these 
types of corrections. 


Accept/Reject Decisions With Sequential Testing 

A critical milestone occurs in product manufacturing at 
delivery time. An ethical producer is concerned about shipping 
a product lot that does not meet specifications. The consumer 
is concerned about spending money to purchase a product that 
does not meet specifications. A test method that permits each 
to have an opportunity to obtain data for decision making is 
required. 

Sequential testing constraints.— If a is the producer’s risk 
and is die consumer’s risk, two delivery time constants valid 
for small risks have been defined and are given as 


A = 


1 -8 


a 


B = 


1 — a 


Let P\ be the probability that Nf failures will occur in time 
t for a specified minimum acceptable fj, and let ?o be the 
probability that N { Mures will occur in time t for an 
arbitrarily chosen upper value fe. Test rules using these four 
constants have been defined for each condition (refs. C-l 
and C-5): 


(1) Accept if P t /P 0 B. 

(2) Reject if P\/Pq s A. 

(3) Continue testing if B < P\/Pq < A. 

Exponential parameter decision making.— As an illustration 
of how these testing constraints can be implemented for the 
exponential distribution, consider example 20. 

Example 20: A purchased quantity of 100 000 tantalum 
capacitors has been received. Negotiations prior to placement 
of the order had established that a - 8 * 0.1, f| = 1000 
hours, and fe « 2000 hours and that the sequential reliability 
test should be truncated in 48 hours. 


(1) Calculate A and B. 

(2) Write the expressions for P 0 and P\ • 

(3) How many units should be placed on test? 

(4) Plot a sequential reliability control graph to facilitate 
decision making at each failure time. 

Solution 20: 


(1) The delivery time constants are obtained by substituting 
values into the defining equations. 


1 - 0.1 


0.1 


-9 


i J UJ U BE TO !V 




B = —^4— = 0.111 


1 - 0.1 


(2) Using binomial distribution from figure C-l and 
substituting values gives P 0 (N f ) and P t (N f ) as 


Po(Nf) 


, \ N f e ~H 2000 


(—) 

\2000j 




Wf) 


= (-) 

\ 1000 / 


ty e -»/iooo 


N/. 


(3) Delivery constant B detines the acceptance criteria for 
P\/Po- Using this constraint and substituting for P, and P 0 
gives 


g _ Pi(ty) _ y N ! g-irtooo 
Po(Nf) 


The minimum testing time without Mure r(0) min is given by 
0.111 * (2)° e -,<0> ” l - /20 ° 

Solving for r(0) min gives 

t(0) m | n = 2.20 x 2000 * 4400 unit-hours 


The minimum number of capacitors to be life tested for 48 
hours is given by 


4400 unit-hours „ _ 
«mi» = — — = 91.7 


48 hours 


To ensure good results, choose a sample size n that is more 
than twice n m | n , for this problem use n «=> 200 units. The 
required minimum testing time for 200 units is given by 


l(m 4400 unit-hours „ . . 
f(0)min “ — rrr — : 22.0 hours 


200 units 


The test can be stopped and an accept/reject decision made 
at t„ where t, is given by - x 


t, * 48 hours x 20 units « 9.6 x 10 3 unit-hours 


(4) The tantalum capacitor reliability chart is constructed 
by using five points in the (N fi t) plane; three of these points 
have already been calculated and are given by 



'( <>)„*,- 4400, ty«0 
t, = 9.6xl0 3 , N f = 0 
/ = 0, N f = 0 


The remaining two points are calculated by using the test 
inequality given by 


B < p(N f ) < A 

In general terms the ratio p(N f ) is given by 

S7\N, 


p{Nf) = Qy ’ e ~On x -\ii t )t 


Taking natural logarithms of the inequality and substituting 
gives 


ln»<ty!n (S)-(i-i)(cta 


Adding (l/f| — l/t Q )t to each term gives 


' nB+ (rr) < ' / H^) <,nA+ (rz)' 


Dividing all terms by ln(^/rj ) gives 


In B 


Inf ^ 


I_I 

i\ k 


Inf -=r 


t < Nf 


< -l2-4 — + 


Inf -i? 


I_i 

h *o 


Inf -i? 


The inequality is now in the form given by 

a + bt < Nf < c + bt 


The constants a and c for this problem for zero failures are 
given by 
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I8E3ETX 




■ -3.18, N f = 0 


the slope b is given by 


(i-i') 

Vi V 5xl0" 4 




* 7.22 x 10 * 4 


Because these boundary constraints are straight lines in the 
form 


N f =bt+ (a or c) 


Figure C 12 shows the resulting tantalum capacitor 
reliability chart. The tantalum capacitor acceptance reliability 
test results in an “accept, ’’.“continue to test,” or “reject” 
decision depending on the failure performance of the capacitors 
as a function of operating time in unit-hours as zoned in 
figure C-12. 



Accept 


If truncated 
along these 
lines 



/- (-3.18,0) 


■'(oWnfO.e.ao) 


-r f (0, 9.6X10 3 ) 


Operating time, t, unlt-hr 


Figure C-12. — 1 Tantalum capacitor reliability cltart. 








Binomial parameters decision making.— For the binomial 
frequency function the procedure to set up a sequential 
reliability test is similar to the Poisson methodology. Because 
the unreliability, or number of defectives, is given by 1 - 17 
for an effectiveness of R, then P\(N f ) is given in binomial 
form by 

^ = (1 


i where 


r .. i 


N, 

? N, 


N, + N f 

number of successful trials 
number of failed trials 


’ Rq, R\ chosen reliability values at some time t. Rq > R { 

i 

| The ratio P\ (N f )/P 0 (Nf) is given by 
.) PiW f ) m (l-jf,)^/?,)"-^ 

Po(N f ) (l - Ro) Nf (Ro) n ~ Nf 

\ 

l Following the steps given in example 20, give four of the 
points in the (N f , t) plane. 

m :ow — n ,= o 

( ln( — ) 

w 

l 

j The test can be stopped and an accept/reject decision made 
| at the number of test truncation 'rials N r ; N r is given by 

i 

j N r — t/N c , N f = 0 

i 

i 

where N c is the number of units chosen for testing. 
n-0, N f =0 


" ■ i 




In B 


In 


*.(!-*>) 


, Nf = 0 


In A 


In 


^6(1 - J? ( ) 


The slope b is given by 




In 

b ~ i„ *><* -*»> 

*i(l -Ro) 



The inequality equation for these conditions is given by 


a + bn < Nf <c + bn 

Accept/reject charts at delivery milestones when based on 
reliability sequential testing methods provide a rigorous mathe- 
matical method for deciding whether or not to accept or reject 
an order of components. The actual reliability value for these 
components is not known, nor is it wise to consider reliability 
assessment at this critical milestone. 

Subsample f chart. —The chief advantages of a subsample 
/chart are (1) it reduces reliability acceptance testing costs, 

(2) it provides for product improvements, (3) it determines 
if statistical control exists, and (4) it determines the mean time 
to repair. 

Example 21: A power supply has the following data: 

(1) Acceptable reliability level, r,, 0.01 failure/hour; 
producer’s reliability risk, R a , 10 percent; specified mean 
time to repair, 3.0 hours 

(2) Lot tolerance fractional reliability deviation, r 2 , 0.005 
failure/hour; consumer’s reliability risk, Rg, 10 percent 

The product test data are given in table C-9. Use figure C-13 
to analyze these data; then answer the following questions: 

(1) What is a suitable time sample and rejection number 
for meeting the 80-percent confidence level selected by 
management? 

(2) What are the subsample sizes and rejection numbers? 

(3) What are the confidence levels for the various rejection 
numbers? 

(4) What arc the control limits on the mean time to repair? 

(5) Plot these data on a subsample /chart. 

(6) What should be done with the manufactured units? 

Solution 21: Given the product data, follow these steps: 

(1) Calculate the confidence level y, the ratio of acceptable 
reliability level to lot tolerance fractional reliability deviation 
k, and the mean time between failures m: 

y » 1 - (Ra + R$ « 1 - (0.1 + 0.1) ■ 0.80, or 80 percent 

k _ r 2 _ 0.005 

" r, " 0.001 " 

m * — = — - — ? * 1000 hours 
r, lxlO" 3 
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TABLE C-9. -POWER SUPPLY PROBLEM DATA 


Sample 

serial 

number 

Number 

of 

failures 

Reason for failure 

Repair 

time, 

hr 

1 

1 

A1A-2VR3 zencr shorted 

1.2 


1 

Ground wire broke 

1.4 


2 

A1A2-VR3 zener shorted; 

5.5. 7.3 



A1A2-Q2 transistor shorted 



0 

In a 250-hr test no failure occurred 


2 

0 

In a 250-hr test no failure occurred 



1 

A3A1-C3 capacitor leaded 

9.5 

3 

1 

A3A1-C3 capacitor leaked 

9.0 


0 

In a 250-hr test no failure occurred 


4 

1 

A7A1-VR1 unsoldered joint 

.5 



A3A1-C3 capacitor leaked 

9.5 

5 

0 

In a 250-hr test no failure occurred 

— 


Looking up Z„ in a normal curve area table (table 3 in ref. 
C-3) for R a = 0.1 shows that Z a « - 1 .28. The value of K 2 
when k = 5 and y - 0.80 is obtained from figure 11-1 in 
reference C-3, where K 2 * 1.05. The equation for t is thus 
' = '”* =■ (1000)(1.05) * 1050 hours a 1000 hours. TTte 
rejection number R for a time sample of 1000 hours and a 
confidence level y - 0.80 is given by 


*1000(0.80) " K 2 + ZJC + 0.5 

* 1.05 + (1.28) 1.025 + 0.5 - 2.86 - 3 


( 2 ) Recalculate the subsample for 7 - 0.50 and* = 5 : From 
figure 11-1 in reference C-3, K 2 * 0.29. Therefore, 


1 - mK 2 = (1000)(0.29) - 290 hours - 250 hours 
Looking up Z a in table 3 in reference C -3 for 


*«-~^-~« 0.25 
2 2 


shows that - -0.6b. Recalculate the rejection number as 
*230(0.30) m K 2 + ZJC + 0.5 

- 0.29 + (0.68) 0.54 + 0.5 
■ 1.16 » 1 failure 



TABLE C-10. -SUBSAMPLE DATA 


t 


y* 

percent 


2. 

*,(y) 

250 

0.25 

0.46 

0.27 

0.61 

1 

500 

.50 

.63 

.185 

.89 

2 

750 

.75 

.73 

.133 

1.11 

2 

1000 

1.0 

.78 

.11 

1.22 

3 


1 


v 4 


(3) Calculate K 2 for each value of / shown in table C -10 as 
„2 _ ' 250 

A “ „ “ MOO * °' 25 for * = 5; m = 1000 hours 


> 

-x,. 


v X "■) 


Look up in figure 1 1-1 in reference C-3 the confidence level 
y values shown in table C-10. Calculate R a for each 
confidence level. (The calculated values are shown in 
table C-10.) 




i 


R _ 1-7 1-0.46 

R a - - — = 0.27 




Look up Z a for each confidence level in table 3 in reference 
C-3 (the values are tabulated in table C- 10 ). Recalculate the 
rejection numbers R, ( 7 ) for each subsample (the values are 
listed in table C- 10 ). 


R,(y) = K 2 + ZJC + 0.5 

*2sxo.4«) = 0.25 + (0.61) 0.5 + 0.5 = 1.05 * 1 
*soo<o.63) = 0.50 + (0.89) 0.71 + 0.5 - 1.63 » 2 
*730(0.73) * 0.75 + (1.11) 0.87 + 0.5 « 2.21 » 2 
*iooo(o.78) ■ 1.00 + (1.22) 1 + 0.5 * 2.72 * 3 


(4) Find the control limits on the mean time to repair for 
the data given in table C-9. 


UCL* 


3# 2X4X3 


X 1(0.( 


«0) 


LCL* 


V* 


3.49 

2x4X3 


■ 6.88 hours 


xim. 


10 ) 


13.4 


1.79 hows 


where/is the average number of failures and 6 denotes mean 
time to repair .These control limits are shown in figure C-13 
for the repair time process. The lower control limit in this 
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TABLE C- II. -POWER SUPPLY ANALYZED DATA 
[Sample size, 230 hr.] 


Time 

Sample 

Subsample 

Reason for failure 

Number of 

Repair 

Mean time 

sample 

serial 

number 


failures 

time, 

to repair, 


number 




hr 

hr 

1 

1 

1 

AIA2-VR3 zener shorted 

1 

1.2 




2 

Ground wirt broke 

1 

1.4 

— 



3 

A1A2-VR3 zener shorted; 

2 

3.5 t 7.3 

5.1 




AHV2-Q2 transistor shorted 






4 

No failures occurred 

0 

— 

— 

2 

2 

3 

No futures occurred 

0 

, ... „ 




4 

A3A1-C3 capacitor leaked 

1 

9.5 

— 


3 

7 

A3AI-C3 capacitor leaked 

1 

9.0 

4.6 



8 

No failures occurred 

0 

— 

— 

3 

4 

9 

A7At-VRl unsoldered joint 

1 

0.5 

— 



10 

A3A1-C3 capacitor leaked 

1 

9.5 

— 


5 

11 

No failures occurred 

0 






12 

No failures occurred 

0 

— 

— 

Totals 

8 

48.9 

1 

— 


has no importance other than statistical completeness because 
any value less than 1.79 hours is an indication of a better 
maintenance activity than wh.it has been specified— a desirable 
condition. 

The completed subsample / chart is shown in figure C-13. 
Table C-ll shows the tabulated data calculated to solve this 
problem. During the various subsample intervals some useful 
conclusions can be drawn. 

(1) During subsample interval 1 to 4 failures 

E/as* 

/-I 

reject serial number 1, request an engineering investigation, 
and repair and retest serial number 1 later. 

(2) During subsample interval 5 to 8 failures 

8 

E f,*R 

t m .5 

ship serial numbers 2 and 3 after all failures have been 
reviewed, the cause identified, and appropriate corrective 


action worked out and approved by an engineering review 
board. 

(3) During subsample interval 9 to 12 failures 

12 

E /si? 

1-9 

ship serial numbers 4 and 5 after all failures have been 
reviewed, properly closed out, and approved by the 
engineering review board. 
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